sanoid
Seaweed File System
Our great sponsors
sanoid | Seaweed File System | |
---|---|---|
118 | 49 | |
2,872 | 14,960 | |
- | - | |
7.1 | 9.9 | |
5 days ago | over 1 year ago | |
Perl | Go | |
GNU General Public License v3.0 only | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
sanoid
- ZFS took full backup (send/receive) without snapshots but now what to prevent another full transfer?
-
ZFS for Dummies
I’m on the other end of the spectrum. I like knowing the flags and settings I use to create the pools.
For snapshots and replication take a look at sanoid (https://github.com/jimsalterjrs/sanoid).
-
Automatic container snapshots? cv4pve or zfs for a small homelab?
I use a combo of PBS and ZFS with Sanoid. The ZFS snapshots are nice because you define it all in a config file with frequent, hourly, daily, weekly, monthly snaps retained however you want. Importantly I use that on the root pool as well as my data pools. With PBS I have a custom retention config and have it backing up VMs and LXCs every couple hours. The PBS dedupe functionality makes it possible to keep full backups going back ages with minimal storage consumption, just the changed blocks. PBS is running inside a VM yes, but it’s storage is on another pool.
- [QUESTION] Dataset level replication
-
YSK: You can 'zfs send' over SSH to the cloud
[3] https://github.com/jimsalterjrs/sanoid/
-
New to ZFS, what layout to choose for 4x20TB drives?
No, it's written by the developer of Sanoid and Syncoid. The foremost open source ZFS snapshotting tool. He contributes on this sub all the time.
- Python Port of 600 Line Bash Script: rsync-time-machine.py for Rsync Backups
-
Some questions about the limitations of ZFS send.
Hi there! Not so much to answer your questions, but are you aware of Sanoid and syncoid? It does backup and remote Sync in the most seamless way. Perhaps this would solve some of your concerns...
- Help me understand zfs send for incremental send AND replication?
- Ask HN: Why isn't BTRFS the default FS in home-oriented Linux distributions?
Seaweed File System
- An open-source distributed object storage service
-
Moving to github.com/seaweedfs/seaweedfs
FYI: Planning to move from github.com/chrislusf/seaweedfs to github.com/seaweedfs/seaweedfs in the coming days. It may cause some problem for package reference, building, documents, and links. Sorry for the change!
-
S3 Isn't Getting Cheaper
Besides storage itself, S3 API access cost can be high if frequently accessed. And latency is unpredicatble.
You can use SeaweedFS Remote Object Store Gateway to cache S3 (or any S3 API compatible vendors) to local servers, and access them at local network speed, and asynchronously sync back to S3.
https://github.com/chrislusf/seaweedfs/wiki/Gateway-to-Remot...
- ### Release 3.12 · chrislusf/seaweedfs
-
Minio in production
If you are looking at MinIO you might find SeaweedFS interesting as well.
- SeaweedFS and YDB
-
Cost effective managed key-value store?
I believe what you want is a horizontally scalable object store with tiered storage. SeaweedFS is free / open source https://github.com/chrislusf/seaweedfs
- A way to store and query large (up to 1GB) user defined objects.
-
Question: does anyone know Storage Provider with S3 as persistence layer?
I don't know if it fits all of your requests, but you can take a look at seaweedfs, which is pretty good
-
Introducing Garage, our self-hosted distributed object storage solution
Seaweedfs deserves a mention here for comparison as well.
What are some alternatives?
zrepl - One-stop ZFS backup & replication solution
minio - The Object Store for AI Data Infrastructure
zfs-auto-snapshot - ZFS Automatic Snapshot Service for Linux
Ceph - Ceph is a distributed object, block, and file storage platform
znapzend - zfs backup with remote capabilities and mbuffer integration.
GlusterFS - Web Content for gluster.org -- Deprecated as of September 2017
zfs_autobackup - ZFS autobackup is used to periodicly backup ZFS filesystems to other locations. Easy to use and very reliable.
MooseFS - MooseFS – Open Source, Petabyte, Fault-Tolerant, Highly Performing, Scalable Network Distributed File System (Software-Defined Storage)
zfs - OpenZFS on Linux and FreeBSD
Apache Hadoop - Apache Hadoop
UrBackup - UrBackup - Client/Server Open Source Network Backup for Windows, MacOS and Linux
lizardfs - LizardFS is an Open Source Distributed File System licensed under GPLv3.