Top 7 erasure-coding Open-Source Projects
-
seaweedfs
SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, cross-DC active-active replication, Kubernetes, POSIX FUSE mount, S3 API, S3 Gateway, Hadoop, WebDAV, encryption, Erasure Coding.
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
bef
Block Erasure Format - An extensible, fast, and usable file utility to encode and decode interleaved erasure coded streams of data.
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
Project mention: DwarFS – The Deduplicating Warp-Speed Advanced Read-Only File System | news.ycombinator.com | 2024-04-11Whoops: WebDAV:
https://news.ycombinator.com/item?id=39417503
SeaweedFS supports WebDAV. https://github.com/seaweedfs/seaweedfs/wiki/WebDAV
I'm not able to find if both/restic supports mounting backups as WebDAV, but in theory there's nothing stopping you.
It's 100% user space (expose a rest service) and supported by a bunch of file-browsers with a bit of a network aware component to it as well.
curl --silent --remote-name --location https://github.com/ceph/ceph/raw/octopus/src/cephadm/cephadmchmod a+x cephadm./cephadm bootstrap --mon-ip 192.168.1.41
Project mention: Tell HN: ZFS silent data corruption bugfix – my research results | news.ycombinator.com | 2023-12-06https://github.com/gluster/glusterfs/issues/894
And apparently apart from modern coreutils using that, it is mostly gentoo users hitting the bugs in lseek.
Project mention: Show HN: Bef – a tool that encodes/decodes interleaved erasure coded streams | news.ycombinator.com | 2024-03-09
erasure-coding related posts
Index
What are some of the best open-source erasure-coding projects? This list will help you:
Project | Stars | |
---|---|---|
1 | seaweedfs | 21,076 |
2 | Ceph | 13,259 |
3 | GlusterFS | 4,498 |
4 | cubefs | 4,315 |
5 | lizardfs | 946 |
6 | infectious | 188 |
7 | bef | 55 |
Sponsored