DwarFS – The Deduplicating Warp-Speed Advanced Read-Only File System

This page summarizes the projects mentioned and recommended in the original post on news.ycombinator.com

Our great sponsors
  • WorkOS - The modern identity platform for B2B SaaS
  • InfluxDB - Power Real-Time Data Analytics at Scale
  • SaaSHub - Software Alternatives and Reviews
  • dwarfs

    A fast high compression read-only file system for Linux, Windows and macOS

  • It's in the man page[1], you put the file at the filesystem creation:

    $ mkdwarfs -i /path/dir -o image.dwarfs

    [1]: https://github.com/mhx/dwarfs/blob/main/doc/mkdwarfs.md

  • dwarffs

    A FUSE filesystem that allows tools like gdb to look up debug info files via HTTP

  • Not to be confused with dwarffs – https://github.com/edolstra/dwarffs.

  • WorkOS

    The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

    WorkOS logo
  • setup

  • seaweedfs

    SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, cross-DC active-active replication, Kubernetes, POSIX FUSE mount, S3 API, S3 Gateway, Hadoop, WebDAV, encryption, Erasure Coding.

  • Whoops: WebDAV:

    https://news.ycombinator.com/item?id=39417503

    SeaweedFS supports WebDAV. https://github.com/seaweedfs/seaweedfs/wiki/WebDAV

    I'm not able to find if both/restic supports mounting backups as WebDAV, but in theory there's nothing stopping you.

    It's 100% user space (expose a rest service) and supported by a bunch of file-browsers with a bit of a network aware component to it as well.

  • kopia

    Cross-platform backup tool for Windows, macOS & Linux with fast, incremental backups, client-side end-to-end encryption, compression and data deduplication. CLI and GUI included.

  • I think Kopia would be great for your use case

    https://kopia.io/

    It has a great system to snapshot files but only store data if it's changed. I use it in an environment where I can't use something like zfs to snapshot data because I don't have the ability to make decisions about what filesystem we're using. It's been amazing, love it so much!

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo
NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts