casync
blobcache
Our great sponsors
| casync | blobcache | |
|---|---|---|
| 17 | 6 | |
| 1,459 | 22 | |
| 0.5% | - | |
| 2.4 | 1.6 | |
| 4 months ago | 11 months ago | |
| C | Go | |
| - | GNU General Public License v3.0 only |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
casync
-
Tool to clone file structure without the large files themselves?
You probably want casync.
-
LibSQL – a fork of SQLite that is both Open Source, and Open Contributions
(personally, I think more people need to be aware of casync for the update storage/distribution problem. It isn't perfect for every use case, but it's good enough that you're probably better off wrapping/forking it rather than reimplementing it badly from scratch)
-
improving download infra
Does something like casync (https://github.com/systemd/casync or https://github.com/folbricht/desync) serve any purpose or provide any advantage to propagating rpm changes over rsync?
-
Are there any true alternatives to Seafile? (Nextcloud is not an alternative in this context)
Software that comes to mind for syncing lots of small files: git (and other source versioning tools), casync (https://github.com/systemd/casync) and a go implementation (https://github.com/folbricht/desync). Not really an answer and I can't think of a way to shoehorn that into your workflow, but maybe it leads you down a useful road.
-
Hacker News top posts: Apr 23, 2022
Casync – A Content-Addressable Data Synchronization Tool\ (15 comments)
-
Casync – A Content-Addressable Data Synchronization Tool
I was wondering how this gets any common chunks at all with the removed file boundaries. Turns out that chunks don't have a set size, just min/max/avg values, so unaligned streams may end up synchronizing. https://github.com/systemd/casync/blob/master/src/cachunker.... If I understood that correctly, that's pretty cool.
But looking at the code I'm having strong "nope" feelings. First, because of lines like "q += m, n -= m;". Second, because of int/enum/semantic abuse: `compression_type` may be _CA_COMPRESSION_TYPE_INVALID which I hope is 0, `>= 0` as a known compression type, or `-EAGAIN` as an error. (from https://github.com/systemd/casync/blob/99559cd1d8cea69b30022... ) I'd bet that just throwing afl at the decompressor will find issues :(
I do like the idea though.
-
Blobcache is a content addressed data store, designed to be a replicated data layer for applications.
Compare https://github.com/systemd/casync which handles splitting/diffing, but does not handle fancy replication.
- Deduplicating Archiver with Compression and Encryption
blobcache
What are some alternatives?
kopia - Cross-platform backup tool for Windows, macOS & Linux with fast, incremental backups, client-side end-to-end encryption, compression and data deduplication. CLI and GUI included.
tarsnap - Command-line client code for Tarsnap.
zstd - Zstandard - Fast real-time compression algorithm
desync - Alternative casync implementation
magic-trace - magic-trace collects and displays high-resolution traces of what a process is doing
BorgBackup - Deduplicating archiver with compression and authenticated encryption.
rclone - "rsync for cloud storage" - Google Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Wasabi, Google Cloud Storage, Yandex Files
restic - Fast, secure, efficient backup program
others - Exhaustive list of backup solutions for Linux
bupstash - Easy and efficient encrypted backups.
bazel-remote - A remote cache for Bazel
spacedrive - Spacedrive is an open source cross-platform file explorer, powered by a virtual distributed filesystem written in Rust.