synbak
tummy-backup
synbak | tummy-backup | |
---|---|---|
2 | 1 | |
20 | 14 | |
- | - | |
4.5 | 0.0 | |
5 months ago | over 2 years ago | |
Shell | Python | |
GNU General Public License v3.0 only | GNU General Public License v3.0 only |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
synbak
-
A almost perfect rsync over SSH backup script
My go-to backup solution, that can also manage rsync over ssh (among plenty other things), is synbak[0]. Short wrapper script to automatically mount the backup medium and it's really simple to (automatically) run a backup job. I use encrypted (LUKS) USB drives for that at home. Highly recommend it.
For encrypted backups to e.g. a NAS, Duplicity[1] is my go-to choice (full backup every month or so, with incremental backup every day inbetween).
[0]: https://github.com/ugoviti/synbak
- Synbak – Universal Backup System
tummy-backup
-
A almost perfect rsync over SSH backup script
I've been doing rsync-based backups of close to a thousand systems for ~20 years, most notably for a long time I backed up the python.org infrastructure, and I have quite a few thoughts on this. I also have a battle-tested rsync wrapper that I'll point to below.
- Backups should be automatic, only requiring attention when it is needed. This script philosophy seems to be "Just do your best, mail a log file, and rely on the user to figure out if something didn't work". Even for home backups, this is just wrong.
- As an example of the above: This script notes that it fails if a backup takes more than 24 hours.
- The "look for other rsyncs running" part of the code is an odd way of approaching locking, but for a single personal "push" backup I guess it is ok.
I've got an rsync wrapper that has been battle tested over a couple decades and hundreds of servers here: https://github.com/linsomniac/tummy-backup/blob/master/sbin/...
Features of it are:
- As the filename implies, the goal is to rsync to a zfs destination, and it will take a zfs snapshot as part of this. It is easy to customize to another backup destination, I've had people report they have customized it for their own laptop backups, for example to an rsync.net destination.
- It goes out of its way to detect when rsync has failed and log that.
- It does do "inplace" rsyncs, which dramatically save space if you have large files that get appended to (logfiles, ZODB databases).
- This is part of a larger system that manages the rsyncs of multiple systems, both local and remote. Things like alerting are done if a backup of a system has failed consistently for many days.
- In the case that there are no failures, there is no e-mail sent, meaning the user only gets actionable e-mails.
The hardlink trick only works for fairly small data sets. Issues include: Managing hard links takes a lot of overhead, especially on spinning discs. Large files being appended to use a ton of space (a 4GB file with 1K appended every day uses 128GB to store 14 dailies, 6 weeklies, and 12 monthlies). ZFS is a pretty good destination for rsync, as similar snapshots will use 4GB to store.
What are some alternatives?
Backupninja
kopia - Cross-platform backup tool for Windows, macOS & Linux with fast, incremental backups, client-side end-to-end encryption, compression and data deduplication. CLI and GUI included.
Elkarbackup - Open source backup solution for your network
restic - Fast, secure, efficient backup program
Backuppc - BackupPC is a high-performance, enterprise-grade system for backing up to a server's disk.
ohmycron - Run cron jobs in a standardized environment with logs and locking
Rsnapshot - a tool for backing up your data using rsync (if you want to get help, use https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss)
Bup - Very efficient backup system based on the git packfile format, providing fast incremental saves and global deduplication (among and within files, including virtual machine images). Please post problems or patches to the mailing list for discussion (see the end of the README below).
Lsyncd - Lsyncd (Live Syncing Daemon) synchronizes local directories with remote targets
Duplicity - Unnoficial fork of Duplicity - Bandwidth Efficient Encrypted Backup
Shield - A standalone system that can perform backup and restore functions for a wide variety of pluggable data systems
UrBackup - UrBackup - Client/Server Open Source Network Backup for Windows, MacOS and Linux