ccheck
arq_restore
ccheck | arq_restore | |
---|---|---|
5 | 124 | |
26 | 647 | |
- | - | |
0.0 | 0.0 | |
about 3 years ago | about 2 years ago | |
Perl | C | |
MIT License | - |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
ccheck
-
Anyone know what causes intermittent corruption of random visual media files across drives and machines?
Grab a friends computer and amass a large batch of good known files, make sure they are of all different file formats. I am pretty sure you will be able to find entire archives of test data in different formats online, to really reproduce this I am going to assume it should be multiple gb in size. Make sure it contains jpg, videos, text files, pdfs, etc. Now write a script or use some tool like this (https://github.com/jwr/ccheck) to basically compute the sha256 checksum of every file in this test package and write it to a file. Take this package of files and copy them to as many media sources as you have access to, CD/DVDs are great, thumb drive, your laptop, a nas with ZFS (and ECC ram) would be amazing, probably throw it up on cloud storage just to be safe. I would then have the same script run as a cron job, maybe on your main machine to basically continuously check that checksums match their original value. As soon as you notice a checksum mismatch you will want to isolate that file and locate the same one across all the other systems and do a deeper inspection. Open it up in a HEX editor and do a bit by bit comparison to see were the corruption occurred and how bad it is. This will start to give you a better picture of what may be going on.
-
Show HN: Off-site, encrypted backups for $1/TB/month at 99.999999999% durability
Here's my "me too" — I've been happily using rclone for things like photo archives (together with my small consistency checker to check file hashes for corruption https://github.com/jwr/ccheck). I also use Arq Backup with B2 as the destination. This gives me very reasonable storage costs and backups I can access and test regularly.
-
What Happened to Perl 7?
Perl is very well suited for certain tasks (not large software systems, but programs that process data). It is also one of very few languages/ecosystems where you can expect your code to work after >10 years. This is why I sometimes use it, for example my fs consistency checker (https://github.com/jwr/ccheck) was written in Perl specifically because it's a long-term tool and I would like to be able to run it on any system in 15 years.
Compare this long-term approach with the fires in Python or (heaven forbid) Node ecosystems, where things break all the time.
-
I Nearly Lost the Lightroom Catalog with All My Photos
This sort of thing scares me. It's why I started running consistency checks on my important archives (like my photo library), which I keep backed up in multiple places. We tend to think that in a digital world bits are just bits and do not get corrupted — which is decidedly untrue.
I wrote my own consistency checker, as I wasn't happy with what was out there. I wanted it to be simple, and maintainable in the long term (>10 years horizon). See https://github.com/jwr/ccheck if you need something like this. I now update my checksums regularly and check for corruption.
-
How do I safely store my files?
Good point about bitrot. This is why I wrote ccheck.pl (https://github.com/jwr/ccheck) — I wanted to be able to check and detect bitrot in a way that depends on as little technology as possible.
arq_restore
-
Ask HN: Open-source Windows 11 backup solutions
There's also Arq Backup which can backup to OneDrive so if you have less than 1TB to backup you might not need to pay anything.
https://www.arqbackup.com/
-
Back Up Storage options
I use Arq Premium backup services They supply the backup software and the cloud storage
-
ThinMachine – A $25 Thin Client macOS Time Machine Appliance
People who want something similar may want to look at Arq [1]. Similar to restic, it provides incremental encrypted backups to most cloud providers (or a machine with SSH). But it is a Mac app, making it easy to configure and maintain. I never had issues with data corruption so far.
Disclaimer: I am not affiliated with them, just a happy user for 9 years.
[1] https://www.arqbackup.com
-
What am I looking for!?
Sending it to the NAS should create a copy automatically to BOTH an online server (I was thinking dropbox, but the space might be too limited. Maybe AWS? I've used https://www.arqbackup.com/ on my computer before so something similar would be great) and a second drive (so I guess I'm looking at a 2-drive NAS that I can setup as master-slave).
-
Guys I’m scared 😭 my life is on my computer and I don’t know how to fix it
For actual off-site backups, using something like https://www.backblaze.com or use https://www.arqbackup.com and your own choice cloud storage provider.
-
Anyone use Veeam Backup for Mac?
Curious too. On the Mac side, I feel like I mostly hear about Backblaze for cloud backups, but I do like Arq for pushing to any of the typical cloud storage options.
-
Cloud backup of TM backups
Have a look at Arq backup https://www.arqbackup.com/ It might meet your requirements
-
Is VeraCrypt 1.25.9 compatible with macOS Ventura?
Take a look at Arq Backup - not identical, but it's E2EE.
-
Could I backup to B2 Cloud with a similar experience as Backblaze Personal?
You could try this: https://www.arqbackup.com/ , it supports B2 I believe.
-
Digital clutter: Learning to let go and stop hoarding terabytes
Regarding Arq backup: if you are worried about using a proprietary (enrypted) and closed-source backup format in case the company were to go under, they have an open source command-line restore tool:
https://github.com/arqbackup/arq_restore
I've been using Arq for years, but I need to look into the "Glacier Deep Archive" format which is about 1/20th the cost of the fastest storage class.
What are some alternatives?
glacier_deep_archive_backup - Extremely low cost, off-site backup/restore using AWS S3 Glacier Deep Archive
restic - Fast, secure, efficient backup program
voidvault - Bootstrap Void with FDE
icloud_photos_downloader - A command-line tool to download photos from iCloud
darktable - darktable is an open source photography workflow application and raw developer
kopia - Cross-platform backup tool for Windows, macOS & Linux with fast, incremental backups, client-side end-to-end encryption, compression and data deduplication. CLI and GUI included.
App-perlbrew - Manage perl installations in your $HOME
rclone - "rsync for cloud storage" - Google Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Wasabi, Google Cloud Storage, Azure Blob, Azure Files, Yandex Files
berrybrew - Perlbrew for Windows!
Cryptomator - Multi-platform transparent client-side encryption of your files in the cloud
roast - 🦋 Raku test suite
vorta - Desktop Backup Client for Borg Backup