pghoard VS backy2

This story starts with our own PgHoard, a PITR backup tool for PostgreSQL. PgHoard offers several methods to archive the WAL (Write Ahead Log), including pg_receivewal, a small application shipping with PostgreSQL which connects to a PostgreSQL cluster using the physical replication protocol to stream WAL as they are produced, optionally keeping track of the position on the server using a replication slot.

Overbuilt and OTT? Sure... but this works fantastically for my use case. I have current backups of everything except my media library because of the size of it; my VM's are all backed up to my Synology nightly using Backy2, my application data gets dumped to that same Synology NAS nightly as well, and all of that also gets synced to Glacier deep storage once a week using Duplicity. I'm going to be adding a new ZFS array later in the year to replace my Synology and hopefully I'll build it out with enough storage to take my media library as well.

It's also been fun for discovery of new things... new tools and use cases. Being able to use Cephfs is great but also being able to leverage it as native S3 buckets is awesome. Learning how to manage snapshots both in RBD images (for my VM's) and Cephfs is cool, and developing my own scripts to snapshot and replicate critical data to my Synology has been rewarding. There's also some pretty cool tools out there even without being as well supported as ZFS like backy2 for backing up RBD images... again to my Synology with a fun little script.

I've been a DataHoarder for a while, but only a modest ~10TB or so. I finally had the space to set up a rack and some servers, and am setting up a Ceph cluster with a ton of old disks I've accumulated over the years, totaling upwards of 20TB. I would like to still have an offsite and preferably offline backup for this data though, but backing up 20+ TB of data to a single drive is obviously off the table. Is there any other alternative to just deploying another Ceph cluster offsite? I don't want to use cloud storage due to the costs, and I also very much prefer to keep all my data under my own physical control. I was looking at Backy2 for the actual extraction of data and writing it to a destination, but that doesn't seem to support idempotent writes (i.e. take one full object and place it on a single drive). I could theoretically combine drives via LVM, but without additional redundancy (I would probably use raid 1 for that) losing one drive would be disastrous, and I am trying to avoid having to add additional redundancy for backups, considering the main ceph cluster will already have 3 copies of the data on it. I also am wondering if I should avoid using Ceph for the backups, since then all my eggs would be in the Ceph basket so to speak. I would love some advice from some of the folks with larger hoards and how you make backups. Thank you!

Have you tried reaching out to the dev on GitHub? I have before and was able to get some bugs ironed out. https://github.com/wamdam/backy2