ccheck VS roast

Grab a friends computer and amass a large batch of good known files, make sure they are of all different file formats. I am pretty sure you will be able to find entire archives of test data in different formats online, to really reproduce this I am going to assume it should be multiple gb in size. Make sure it contains jpg, videos, text files, pdfs, etc. Now write a script or use some tool like this (https://github.com/jwr/ccheck) to basically compute the sha256 checksum of every file in this test package and write it to a file. Take this package of files and copy them to as many media sources as you have access to, CD/DVDs are great, thumb drive, your laptop, a nas with ZFS (and ECC ram) would be amazing, probably throw it up on cloud storage just to be safe. I would then have the same script run as a cron job, maybe on your main machine to basically continuously check that checksums match their original value. As soon as you notice a checksum mismatch you will want to isolate that file and locate the same one across all the other systems and do a deeper inspection. Open it up in a HEX editor and do a bit by bit comparison to see were the corruption occurred and how bad it is. This will start to give you a better picture of what may be going on.

Here's my "me too" — I've been happily using rclone for things like photo archives (together with my small consistency checker to check file hashes for corruption https://github.com/jwr/ccheck). I also use Arq Backup with B2 as the destination. This gives me very reasonable storage costs and backups I can access and test regularly.

Perl is very well suited for certain tasks (not large software systems, but programs that process data). It is also one of very few languages/ecosystems where you can expect your code to work after >10 years. This is why I sometimes use it, for example my fs consistency checker (https://github.com/jwr/ccheck) was written in Perl specifically because it's a long-term tool and I would like to be able to run it on any system in 15 years.

Compare this long-term approach with the fires in Python or (heaven forbid) Node ecosystems, where things break all the time.

This sort of thing scares me. It's why I started running consistency checks on my important archives (like my photo library), which I keep backed up in multiple places. We tend to think that in a digital world bits are just bits and do not get corrupted — which is decidedly untrue.

I wrote my own consistency checker, as I wasn't happy with what was out there. I wanted it to be simple, and maintainable in the long term (>10 years horizon). See https://github.com/jwr/ccheck if you need something like this. I now update my checksums regularly and check for corruption.

Good point about bitrot. This is why I wrote ccheck.pl (https://github.com/jwr/ccheck) — I wanted to be able to check and detect bitrot in a way that depends on as little technology as possible.

The Raku programming language has, with some tweaks:

data.store.book.grep(.price < 10).map(.title)

Although personally I would write that as:

data.store.book.map: { .title if .price < 10 }

which combines the filter / map into a single operation.

https://raku.org

Add more IO::Path::parent tests #801: merged 2022-02-19

23. Raku - $79,448

Pakku is a package manager for the Raku Programming Language. Latest releases of Pakku are part of Pakku Celastrina version family. Celastrina name means elegant and beutiful, So I will take the opportunity to introduce how elegant IMO Pakku handles Raku distributions.

At the last European Perl Conference I proposed to change the name of "Perl 6". After a lot of discussion, it was decided that it was going to be called the Raku Programming Language.

Raku note: This language has no length method on strings, because in Unicode world it is super confusing. Instead there are separate methods to ask precisely about amount of characters, amount of code points and amount of bytes.

Elizabeth Mattijsen reports on all recent developments around Rakudo, an implementation of the Raku Programming Language.

The raku.org website.