Our great sponsors
-
But if you want to skip the rclone and go straight to backing up to a cloud provider take a look at Duplicacy.
-
While we're talking restic tools, have you also tried restic-profile?
-
InfluxDB
Collect and Analyze Billions of Data Points in Real Time. Manage all types of time series data in a single, purpose-built database. Run at any scale in any environment in the cloud, on-premises, or at the edge.
-
If you want to find and organize duplicates on disk, then there are other tools for that. I'm not particularly familiar with them, but you might try starting with czkawka which looks pretty featureful. The github page also compares it with a couple other similar tools. Otherwise you could try searching for "find duplicates" in /r/datahoarders or /r/selfhosted for ideas.
-
rclone
"rsync for cloud storage" - Google Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Wasabi, Google Cloud Storage, Yandex Files
While your recommendation totally makes sense for this use case, I just wanted to add that s3fs is probably not the way to go with Borg if you're trying to use it with generic cloud object storage—for all the reasons you mention. Instead, my understanding is that Borg users often use rclone or similar to sync a local Borg repository to cloud storage. Much more reliable, but the downside is it requires extra local storage.
