lrzip
Duplicati
lrzip | Duplicati | |
---|---|---|
7 | 22 | |
595 | 10,285 | |
- | 2.2% | |
3.7 | 9.5 | |
24 days ago | 6 days ago | |
C | C# | |
GNU General Public License v3.0 only | GNU General Public License v3.0 or later |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
lrzip
-
How to Get Your Backup to Half of Its Size – ZSTD Support in XtraBackup
lrzip
Long Range ZIP or LZMA RZIP
https://github.com/ckolivas/lrzip
"A compression utility that excels at compressing large files (usually > 10-50 MB). Larger files and/or more free RAM means that the utility will be able to more effectively compress your files (ie: faster / smaller size), especially if the filesize(s) exceed 100 MB. You can either choose to optimise for speed (fast compression / decompression) or size, but not both."
-
File compression
7zip and XZ are almost always the best in any comparison. (They use the same algorithm.) Occasionally something new comes allong that may be bettyer, but it fades away... Like lrzip. https://lkml.org/lkml/2011/6/4/23 https://github.com/ckolivas/lrzip
-
If we found a way to reverse a hashing function, would that make them ultra-compression algorithms?
For example lrzip has an intense "dupe hunting" mode and takes days for large content, but does compress very well once it's done (and expansion is fast). I use it on long term storage backups and disk images and junk. Completely incompatible with streaming, unlike chunk-based like gzip or deflate or etc, although unpacking can stream such as searching or verifying a tarfile archive. But the original source has to be file-based so seeking for the hunting can work across the entire file-as-a-block.
- Lrzip – Long Range Zip or LZMA RZIP
-
Ask HN: How would you store 10PB of data for your startup today?
Best I know of for that is something like lrzip still, but even then it's probably not state of the art. https://github.com/ckolivas/lrzip
It'll also take a hell of a long time to do the compression and decompression. It'd probably be better to do some kind of chunking and deduplication instead of compression itself simply because I don't think you're ever going to have enough ram to store any kind of dictionary that would effectively handle so much data. You'd also not want to have to re-read and reconstruct that dictionary to get at some random image too.
-
Encrypted Backup Shootout
There's also lrzip for large files: https://github.com/ckolivas/lrzip
Duplicati
-
C# hakkında görüşleriniz ve ödevim
Petabaytlarca veri emanet edilen şu kodun %85'i C# https://github.com/duplicati/duplicati
-
Is there a non-beta version?
For my parents' computer, I'm using the canary version (which should be the alpha version I think?) of Duplicati since years.
-
Nextcloud noob: How can I auto backup photos & files to AWS/iCloud?
It hasn't had a release in a little while but work is still occurring. https://github.com/duplicati/duplicati/actions.
- Most used selfhosted services in 2022?
- Backup Windows PC to Minio/S3
-
Announcing Duplicati Dashboard
Hey have a read at : https://github.com/duplicati/duplicati/issues/4041
-
A Dummies Guide to Duplicati
I just came across this while looking through their issues to see if anyone else had reported the firefox issue i'm running into. I'm starting to have serious reservations.
- Apparently you cannot have the Kanye interview on Google Drive now
-
Borg vs Duplicacy (not Duplicati or Duplicity)?
I like duplicacy because of the way it keeps the chunks in the file system, without a special database. This makes it scale up really well no matter how many backups you have (you can even have multiple computers saved). It's kind of beyond weird how you select what you want to backup with the symlinks (using the command line version), looks more like what one would make for himself in a weekend (not that I'm complaining about free software!) but it's been without bugs for me and extremely efficient. In contrast duplicati has a perfect interface, it's well maintained and everything but bogs down in any large backup, has stories about people recovering for weeks for a very few local TBs and I've experienced for myself this, granted in the python that is checking the sha256 checksums of the backups but it makes it slower many times (possibly hundreds of times), nobody checked from 2013 to 2021 (or did it on tiny datasets like 1GB or was content to wait for weeks even on something small-ish)?
- C# library for centralized cloud storage syncing?
What are some alternatives?
bupstash - Easy and efficient encrypted backups.
UrBackup - UrBackup - Client/Server Open Source Network Backup for Windows, MacOS and Linux
rdedup - Data deduplication engine, supporting optional compression and public key encryption.
BorgBackup - Deduplicating archiver with compression and authenticated encryption.
duplicity - mirror of duplicity: https://code.launchpad.net/duplicity
restic - Fast, secure, efficient backup program
LeoFS - The LeoFS Storage System
Duplicity - Unnoficial fork of Duplicity - Bandwidth Efficient Encrypted Backup
rclone - "rsync for cloud storage" - Google Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Wasabi, Google Cloud Storage, Azure Blob, Azure Files, Yandex Files
ParlAI - A framework for training and evaluating AI models on a variety of openly available dialogue datasets.
TimeShift - System restore tool for Linux. Creates filesystem snapshots using rsync+hardlinks, or BTRFS snapshots. Supports scheduled snapshots, multiple backup levels, and exclude filters. Snapshots can be restored while system is running or from Live CD/USB.