lrzip | tarsnap | |
---|---|---|
6 | 11 | |
595 | 848 | |
- | 0.4% | |
3.7 | 8.3 | |
23 days ago | 11 days ago | |
C | C | |
GNU General Public License v3.0 only | GNU General Public License v3.0 or later |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
lrzip
-
How to Get Your Backup to Half of Its Size – ZSTD Support in XtraBackup
lrzip
Long Range ZIP or LZMA RZIP
https://github.com/ckolivas/lrzip
"A compression utility that excels at compressing large files (usually > 10-50 MB). Larger files and/or more free RAM means that the utility will be able to more effectively compress your files (ie: faster / smaller size), especially if the filesize(s) exceed 100 MB. You can either choose to optimise for speed (fast compression / decompression) or size, but not both."
-
File compression
7zip and XZ are almost always the best in any comparison. (They use the same algorithm.) Occasionally something new comes allong that may be bettyer, but it fades away... Like lrzip. https://lkml.org/lkml/2011/6/4/23 https://github.com/ckolivas/lrzip
-
If we found a way to reverse a hashing function, would that make them ultra-compression algorithms?
For example lrzip has an intense "dupe hunting" mode and takes days for large content, but does compress very well once it's done (and expansion is fast). I use it on long term storage backups and disk images and junk. Completely incompatible with streaming, unlike chunk-based like gzip or deflate or etc, although unpacking can stream such as searching or verifying a tarfile archive. But the original source has to be file-based so seeking for the hunting can work across the entire file-as-a-block.
- Lrzip – Long Range Zip or LZMA RZIP
-
Ask HN: How would you store 10PB of data for your startup today?
Best I know of for that is something like lrzip still, but even then it's probably not state of the art. https://github.com/ckolivas/lrzip
It'll also take a hell of a long time to do the compression and decompression. It'd probably be better to do some kind of chunking and deduplication instead of compression itself simply because I don't think you're ever going to have enough ram to store any kind of dictionary that would effectively handle so much data. You'd also not want to have to re-read and reconstruct that dictionary to get at some random image too.
-
Encrypted Backup Shootout
There's also lrzip for large files: https://github.com/ckolivas/lrzip
tarsnap
-
Where do you store your backups? What Provider if any?
Tarsnap for configs and critical stuff (password database, emails).
-
3,200-Year-Old Egyptian Tablet Records Excuses for Why People Missed Work
Someone does :)
https://tarsnap.com
> Tarsnap uses a prepaid model based on actual usage:
> Storage: 250 picodollars / byte-month of encoded data
-
What is the best private encrypted cloud storage?
Colin Percival's tarsnap
-
Ask HN: Codebases with great, easy to read code?
In past threads, people have mentioned enjoying my Tarsnap (https://github.com/Tarsnap/tarsnap) code. I personally think that the spiped (https://github.com/Tarsnap/spiped) code is even better.
-
I love the idea of tarsnap but a stable release hasn't been released since 2017. Is there a modern alternative, or is tarsnap actually still usable and secure?
I prefer Vorta myself ( https://github.com/borgbase/vorta ) as it also has incremental and encrypted backups, as well as being a fraction of the price, but tarsnap seems to still be in very-slow development: https://github.com/Tarsnap/tarsnap , so I'd say from a quick look it's still trustworthy.
- Restic: Backups Done Right
-
What's your backup strategy?
Each server also upload their configs and « important » data (my mails and git repos) to tarsnap 3. Tarsnap storage is not as cheap as B2, so I try not to upload too much data there, but it's reliable and easy to use. It was also my first backup solution, and barely cost me 10$ a year so I keep it as a secondary backup.
-
FreeBSD SSH Hardening
Not foolish! The Tarsnap client code is open source, but the license file prohibits anyone from using the code: https://github.com/Tarsnap/tarsnap/blob/master/COPYING
> Redistribution and use in source and binary forms, without modification,
- Deduplicating Archiver with Compression and Encryption
-
The Wrong Way to Switch Operating Systems on Your Server
Yes. For the curious,
https://github.com/Tarsnap/tarsnap/graphs/contributors
What are some alternatives?
bupstash - Easy and efficient encrypted backups.
BorgBackup - Deduplicating archiver with compression and authenticated encryption.
rdedup - Data deduplication engine, supporting optional compression and public key encryption.
restic - Fast, secure, efficient backup program
duplicity - mirror of duplicity: https://code.launchpad.net/duplicity
rclone - "rsync for cloud storage" - Google Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Wasabi, Google Cloud Storage, Azure Blob, Azure Files, Yandex Files
LeoFS - The LeoFS Storage System
nixos-infect - [GPLv3+] install nixos over the existing OS in a DigitalOcean droplet (and others with minor modifications)
ParlAI - A framework for training and evaluating AI models on a variety of openly available dialogue datasets.
Duplicacy - A new generation cloud backup tool