Top 16 distributed-storage Open-Source Projects

• seaweedfs

  34 21,013 9.9 Go

  SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, cross-DC active-active replication, Kubernetes, POSIX FUSE mount, S3 API, S3 Gateway, Hadoop, WebDAV, encryption, Erasure Coding.

  

Project mention: DwarFS – The Deduplicating Warp-Speed Advanced Read-Only File System | news.ycombinator.com | 2024-04-11

Whoops: WebDAV:

https://news.ycombinator.com/item?id=39417503

SeaweedFS supports WebDAV. https://github.com/seaweedfs/seaweedfs/wiki/WebDAV

I'm not able to find if both/restic supports mounting backups as WebDAV, but in theory there's nothing stopping you.

It's 100% user space (expose a rest service) and supported by a bunch of file-browsers with a bit of a network aware component to it as well.

• Ceph

  34 13,197 10.0 C++

  Ceph is a distributed object, block, and file storage platform

  

Project mention: First time user sturggles | /r/ceph | 2023-06-24

curl --silent --remote-name --location https://github.com/ceph/ceph/raw/octopus/src/cephadm/cephadmchmod a+x cephadm./cephadm bootstrap --mon-ip 192.168.1.41

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• dragonboat

  2 4,936 6.4 Go

  A feature complete and high performance multi-group Raft library in Go.

• cubefs

  1 4,311 9.8 Go

  cloud-native file store

  

• storj

  44 2,869 9.9 Go

  Ongoing Storj v3 development. Decentralized cloud object storage that is affordable, easy to use, private, and secure.

  

Project mention: Do you donate your CPU time, storage, or bandwidth from your homelab to any altruistic purpose? | /r/homelab | 2023-10-16

I did storj.io but was not profitable and the support was worthless. Did join NTP Pool (as I have a stratum 2 GPS NTP) but the power supply died and I haven't been able to get time to fix it.

• Pravega

  2 1,966 8.3 Java

  Pravega - Streaming as a new software defined storage primitive

  

• MooseFS

  3 1,583 4.8 C

  MooseFS – Open Source, Petabyte, Fault-Tolerant, Highly Performing, Scalable Network Distributed File System (Software-Defined Storage)

  

Project mention: Google Cloud Storage FUSE | news.ycombinator.com | 2023-05-02

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

• LeoFS

  2 1,536 0.0 Erlang

  The LeoFS Storage System

  

Project mention: Leofs – S3 / NFS object store | news.ycombinator.com | 2024-04-18

• zenoh

  13 1,243 9.5 Rust

  zenoh unifies data in motion, data in-use, data at rest and computations. It carefully blends traditional pub/sub with geo-distributed storages, queries and computations, while retaining a level of time and space efficiency that is well beyond any of the mainstream stacks.

  

Project mention: Zenoh: Zero Overhead Network Protocol | news.ycombinator.com | 2024-04-24

• SheepDog

  0 978 0.0 C

  Distributed Storage System for QEMU

  

• daos

  1 700 9.8 C

  DAOS Storage Stack (client libraries, storage engine, control plane)

  

• good-karma-kit

  9 293 4.7

  😇 A Docker Compose bundle to run on servers with spare CPU, RAM, disk, and bandwidth to help the world. Includes Tor, ArchiveWarrior, BOINC, and more...

  

• kertish-dfs

  3 106 0.0 Go

  Kertish-dfs is a simple distributed storage platform, implements file storage on a single distributed computer cluster, and provides interfaces for file/folder handling. Kertish-dfs aims primarily for completely distributed operation without a single point of failure, scalable to the exabyte level.

• hedge

  1 22 7.4 Go

  A distributed systems library for Kubernetes deployments built on top of spindle and Cloud Spanner.

• Distributed-Systems-Guide

  1 22 4.2

  Distributed Systems Guide

• whitepaper

  1 3 4.9

  Astreum White Paper (by astreum)

  

• SaaSHub

  www.saashub.com sponsored

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

  

distributed-storage related posts

• Leofs – S3 / NFS object store
  1 project | news.ycombinator.com | 18 Apr 2024
• DwarFS – The Deduplicating Warp-Speed Advanced Read-Only File System
  8 projects | news.ycombinator.com | 11 Apr 2024
• Billion File Filesystem
  2 projects | news.ycombinator.com | 8 Feb 2024
• SeaweedFS fast distributed storage system for blobs, objects, files and datalake
  7 projects | news.ycombinator.com | 2 Feb 2024
• DuckDB + dbt for a serverless event correlation pipeline?
  1 project | /r/dataengineering | 7 Dec 2023
• Do you donate your CPU time, storage, or bandwidth from your homelab to any altruistic purpose?
  1 project | /r/homelab | 16 Oct 2023
• NFT Payload Storage Options
  2 projects | dev.to | 24 Jul 2023
• A note from our sponsor - SaaSHub
  www.saashub.com | 25 Apr 2024

  SaaSHub helps you find the best software and product alternatives Learn more →

Index

Sponsored

SaaSHub - Software Alternatives and Reviews

SaaSHub helps you find the best software and product alternatives

www.saashub.com