Seaweed File System Alternatives

Posts

• I received first-ever donation on my open-source side project and it felt great
  news.ycombinator.com | 2021-01-12

  I am receiving $337 per month as of now for https://github.com/chrislusf/seaweedfs

  Not something to be proud of if including the time spent to evolve the project.

• Ask HN: Where to get cheap VPS with big storage?
  news.ycombinator.com | 2021-01-10

  > Could you provide more details on this? Are you saying accessing the metadata will be a bottleneck (via E.g., a relational database)?

  It's the problem of having too many small files. Explained here https://github.com/chrislusf/seaweedfs#introduction.

  > Also, what do you mean by fill/rebalance in this context?

  Assuming you are replicating the data. You have to be careful of cascading failures if repairing replication is too slow because of slow network/disk-io/cpu.

  news.ycombinator.com | 2021-01-10

  Lowest I've seen is https://www.hetzner.com/dedicated-rootserver/sx62 at 1.6€ / TB / month. Anyone seen lower ?

  You should be extra careful on big servers with little bandwidth, you might need a month to fill/empty/rebalance them.

  How will you host the images ? Metadata will become a bottleneck before hdd size.

  Check out https://github.com/chrislusf/seaweedfs

• Use the distributed file system as a Key-Large-Value store
  news.ycombinator.com | 2020-12-29

  I am working on SeaweedFS, and developed a feature that should be interesting.

  Basically, it enhances a distributed key value store, such as Redis/Cassandra/HBase/MongoDB to a distributed key-large-value store, by storing large values as files to SeaweedFS. The smaller values can be optionally stored in the KV store directly, saving one network hop.

  Let me what you think.

  https://github.com/chrislusf/seaweedfs/wiki/Filer-as-a-Key-Large-Value-Store

• Distributed Key Value Store With Good On Disk
  reddit.com/r/bigdata | 2020-12-29

  I am working on SeaweedFS, in which meta data are stored in most common key value stores, such as Redis/Cassandra/MongoDB/etc. Basically you can use SeaweedFS as a key-large-value store, with configurable replications. You can also configure that the larger values are stored as objects on data nodes, and smaller values are stored in the key value store directly. https://github.com/chrislusf/seaweedfs/wiki/Filer-as-a-Key-Large-Value-Store

  reddit.com/r/Database | 2020-12-29

  I am working on SeaweedFS. You can use it as a key-large-value store directly, with configurable replications. https://github.com/chrislusf/seaweedfs/wiki/Filer-as-a-Key-Large-Value-Store

• Distributed storage with friends
  reddit.com/r/selfhosted | 2020-12-28

  In the mean time I found this: https://github.com/chrislusf/seaweedfs. It looks like it may work, I was going to try this somewhere this week.

• Fast and easily scalable self-hosted storage solution
  reddit.com/r/selfhosted | 2020-12-24

  MinIO seems quite good from my standpoint (single binary, compatibility w/ S3 tooling, etc). My only concern is: it seems to be quite annoying with regards to scaling with heterogeneous clusters. Let's say I currently have 3 servers - one with 8*5TB and 2 with 4*10TB - and I'm about to add 2 new servers with 3*10TB... (https://docs.minio.io/docs/minio-federation-quickstart-guide.html / https://github.com/minio/minio/issues/7411). SeaweedFS's interface seems a lot easier: you can simply join an existing master: https://github.com/chrislusf/seaweedfs#start-volume-servers...

• Gopher Gold #15 - Wed Oct 14 2020
  dev.to | 2020-10-14

  chrislusf/seaweedfs (Go): SeaweedFS is a distributed object store and file system to store and serve billions of files fast! Object store has O(1) disk seek, transparent cloud integration. Filer supports cross-cluster active-active replication, Kubernetes, POSIX, S3 API, encryption, Erasure Coding for warm storage, FUSE mount, Hadoop, WebDAV.