btrbk VS bees

That's still easier using snapshots and something like btrbk. Snapshot the directory at start, prune if there are too many snapshots (or snapshots get too old).

I'm on Arch, but you might still find it useful: Btrfs snapshots Arch Wiki - Incremental backup to external drive GitHub - btrbk

the smoothest backup tool i have seen for Linux is btrbk works real nice and is customizable for almost all use-cases BTRFS rocks :)

I recommend redundancy and regular verification is you want to insure your data against corruption. If you do that, you can forget about things like ECC. My setup is a NUC server running Ubuntu with a USB3-connected storage drive running BTRFS. I use btrbk to auto-snapshot and auto-replicate via incremental sends to my BTRFS backup drive, and RotKraken to track integrity of the data with a monthly verification run so that I notice corruption in time to correct it.

In pondering my backup strategy, I was wondering if I could use BTRFS snapshots and a backup tool like btrbk, which is a nice integrated snapshot/backup solution I've used happily on desktop Linux. BTRFS needs subvolumes for snapshots, so I couldn't backup the host itself (which wasn't installed with a / subvolume like other distributions I've used), but it could snapshot the VMs and containers, which have their own individual subvolumes. Then btrbk can send that snapshot in an incremental fashion to external storage.

I then installed btrbk and tried to follow the instructions to create snapshots of root and home on the SSD and then send/receive those to the HDD. I mainly used https://github.com/digint/btrbk and https://mutschler.dev/linux/fedora-btrfs-35/, but I don't use luks.

For anyone using btrfs on their system, I heartily recommend btrbk, which has served me very well for making incremental backups with a customizable retention period: https://github.com/digint/btrbk

btrbk is the best solution I know.

I use BTRBK to make and copy the BTRFS snapshots to my HDD. I schedule it to run every 3 hours using a Sytemd unit file through my own script to avoid running the backup at inconvenient moments:

Btrfs defrag 'will break up the reflinks of COW data' and 'may cause considerable increase of space usage depending on the broken up reflinks'. To try to fix this, I would run bees to try and deduplicate the now duplicate reflinks. It may be worth doing this from e.g. a livedisk though as out of space errors can cause things to break (so don't upgrade packages till you fix this).

Figuring out which paths are needed outside gcroots'ed closures is pretty complicated. If you're using flakes, the main issue is duplicates, so store optimization and bees may help. With channels, once you update a channel you might as well gc everything else.

bees

I felt more confident with the code/developer/docs. The author knows his stuff regarding btrfs. Like, look at this, it's amazing: https://github.com/Zygo/bees/blob/master/docs/btrfs-kernel.md

I have some time again and wondering if anyone's got Bees, https://github.com/Zygo/bees, running on their Synology.

Even if NixOS doesn't support sending deduplicating syscalls to the kernel, you could use the Btrfs deduping daemon called bees to slowly save space over time. There might be an equivalent for ZFS, too.

This isn't much different than ext4. 0 length files can happen after a crash. You can avoid this by mounting with flushoncommit for the future. See here for details.

Maybe BEES can help you to dedup any blocks, not file.