Fast and easily scalable self-hosted storage solution

• Seaweed File System

  49 14,960 9.9 Go

  Discontinued SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, cross-DC active-active replication, Kubernetes, POSIX FUSE mount, S3 API, S3 Gateway, Hadoop, WebDAV, encryption, Erasure Coding. [Moved to: https://github.com/seaweedfs/seaweedfs] (by chrislusf)

  

MinIO seems quite good from my standpoint (single binary, compatibility w/ S3 tooling, etc). My only concern is: it seems to be quite annoying with regards to scaling with heterogeneous clusters. Let's say I currently have 3 servers - one with 8*5TB and 2 with 4*10TB - and I'm about to add 2 new servers with 3*10TB... (https://docs.minio.io/docs/minio-federation-quickstart-guide.html / https://github.com/minio/minio/issues/7411). SeaweedFS's interface seems a lot easier: you can simply join an existing master: https://github.com/chrislusf/seaweedfs#start-volume-servers...

• sandcrawler

  2 23 0.0 HTML

  Backend, IA-specific tools for crawling and processing the scholarly web. Content ends up in https://fatcat.wiki

  

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

Suggest a related project