snap-sync VS bees

- https://github.com/qubidt/snap-sync

You might be interested in snap-sync. It is a bash script which accomplishes btrfs backups using snapper under the hood. So no need to use btrbk (but you get all the same functionality).

I've been using snapper and snap-sync for automated snapshots and backups to an external drive. Recently, snap-sync is no longer maintained, and it may not be able to do some more things that I want to do, such as:

I was actively using snap-sync to back up my files to a local hard drive. Unfortunately, snap-sync will be retired soon.

I use snapper for snapshots and snap-sync for backing up said snapshots on an external drive.

Yeah, I use snapshots for backups using snap-sync, which is a convenience script around btrfs send. Personally I use it to back up onto an external HDD, but it can also do remote backups over SSH. I'm just not sure if remote backups are incremental or not.

On my laptop I make extensive use of openSUSE's snapper and the snap-sync script to sync to two external USB drives. Lastly, I wrote a script to clean/expire snapshots on the external volumes.

Snapshots aren't proper backups by themselves; they don't protect you from disk failure or the entire filesystem somehow being corrupted. It is possible to use them as backups; personally I use a command-line script called snap-sync to do this. This has the advantage of retaining the incremental nature of snapshots (though the incrementality will be on your backup media, so the first snapshot of a 100GB filesystem you put on your backup drive will take up 100GB of space), plus it integrates nicely with Snapper which is the snapshot utility I use.

Btrfs defrag 'will break up the reflinks of COW data' and 'may cause considerable increase of space usage depending on the broken up reflinks'. To try to fix this, I would run bees to try and deduplicate the now duplicate reflinks. It may be worth doing this from e.g. a livedisk though as out of space errors can cause things to break (so don't upgrade packages till you fix this).

Figuring out which paths are needed outside gcroots'ed closures is pretty complicated. If you're using flakes, the main issue is duplicates, so store optimization and bees may help. With channels, once you update a channel you might as well gc everything else.

bees

I felt more confident with the code/developer/docs. The author knows his stuff regarding btrfs. Like, look at this, it's amazing: https://github.com/Zygo/bees/blob/master/docs/btrfs-kernel.md

I have some time again and wondering if anyone's got Bees, https://github.com/Zygo/bees, running on their Synology.

Even if NixOS doesn't support sending deduplicating syscalls to the kernel, you could use the Btrfs deduping daemon called bees to slowly save space over time. There might be an equivalent for ZFS, too.

This isn't much different than ext4. 0 length files can happen after a crash. You can avoid this by mounting with flushoncommit for the future. See here for details.

Maybe BEES can help you to dedup any blocks, not file.