rest-server VS Snebu

Similar here. Termux with restic, so it does deduplication and encryption and such (also compression since a few months but haven't turned it on yet).

On local laptop: run https://github.com/restic/rest-server/ to accept the incoming data, then (if 1234 is the port that rest-server runs on):

    user@laptop:~$ ssh -R 1234:localhost:1234 root@phone

I use restic to a cloud storage provider and restic-server to another nas. I used hyperbackup for a long time but proved to not be flexible enough and I wanted to get away from a proprietary backup that could only be restored on another Synology.

Have a look at restic and restic-rest-server.

Min.io is just one of the supported storage backends. If you prefer, the restic rest server seems to be supported and might be easier to host. https://github.com/restic/rest-server

restic with rest-server

This one is quite unclear:

> We have added checksums for various backends so data uploaded to a backend can be checked there.

What do you mean checksums? All data is already stored in files with as filename the sha256sum of the contents, so clearly it's all already checksummed and can be verified right?

Looking into the changelog entry[1], this is about verifying the integrity upon uploading:

> The verification works by informing the backend about the expected hash of the uploaded file. The backend then verifies the upload and thereby rules out any data corruption during upload. \n\n [...] besides integrity checking for uploads [this] also means that restic can now be used to store backups in S3 buckets which have Object Lock enabled.

Object lock is mentioned in passing (and only in this more detailed info) but this is a big one. S3 docs:

> Object Lock can help prevent objects from being deleted or overwritten for a fixed amount of time or indefinitely.

i.e. ransomware protection. Good luck wiping backups if your backup host refuses to overwrite or delete the files. And you know the files are good because they match their hash.

Extortion is still a thing, but if people would use this, it more-or-less wipes out the attack vector of ransomware. The only risk is if the attacker is in your systems long enough to outlast your retention period. Did anyone say "test your backups"?

For self-hosting, restic has a custom back-end called rest-server[2] for that which supports a so-called "append-only mode" (no overwriting or deleting). I worked on the docs for this[3] together with rawtaz and MichaelEischer to make this more secure, because eventually, of course, your disks are full or you want to stop paying for legacy data on S3, and an attacker could have added dummy backups to fool your automatic removal script into thinking it needs to leave only the dummy backups. Using the right retention options, this attack cannot happen.

Others are doing some pretty cool stuff in the backup sphere as well, e.g. bupstash[4] has public key encryption so you don't need to have the decryption keys as a backup client.

[1] https://github.com/restic/restic/releases/v0.13.0

[2] https://github.com/restic/rest-server/

[3] https://restic.readthedocs.io/en/latest/060_forget.html#secu...

[4] https://github.com/andrewchambers/bupstash/

The append-only mode can be implemented using https://github.com/restic/rest-server or services like rsync.net that offer read-only zfs snapshots. Doesn’t solve the asymmetric crypto of course.

But how is that better than running the REST server which is also an HTTP-based API? Or is it? I suspect the answer is going to be system dependent but I am curious.

As such, I use tar for the serialization of backup data for Snebu (https://www.snebu.com), which has a plugin (tarcrypt) that operates on the data streams. Snebu ingests tar format, and emits tar format, so all you need to backup/restore a host is ssh access (server can pull backups, or client can push backups). So tarcrypt was added as way to do client-side encryption, but still be able to to submit recognizable tar files to Snebu's backend (which indexes, de-duplicates, and snapshots backups).

Snebu, on github. Simple Network Encrypting Backup Utility.

Details are at https://www.snebu.com/tarcrypt.html if you want to look it over (and tarcypt is part of the Snebu project https://github.com/derekp7/snebu). I'd love to get another pair of eyes on this to point out any non-obvious security limitations.

Since I've had a handful of users ask about cloud storage for Snebu, Would you be interested in adding Snebu as a supported protocol? It should be similar to how you currently support Borg. For Snebu, the client runs find and tar, sending results via ssh to the snebu binary on the remote host. And more recently client-side public key encryption support has been added via a client-side filter called "tarcrypt". Ideally, a customer would use Snebu to back up to a local device on their network (for example a Raspberry Pi with a large USB drive attached), and then use Snebu's efficient replication to send deltas to the cloud-hosted server. Client files are stored individually (deduplicated) on the Snebu server, and metadata is in an SQLite DB (advantages over Borg is more open standards for the data storage and public-key encryption, disadvantage is file-level instead of block-level deduplication and a project that isn't as widely used).

If you are interested, I would be more then happy to have an extended discussion with you going over implementation options, and updating the client side script to make it work better with your service. (https://www.snebu.com, https://github.com/derekp7/snebu, and the tarcrypt extensions to tar are described at https://www.snebu.com/tarcrypt.html).

Would a mention of my open source backup system, Snebu (or https://github.com/derekp7/snebu) fit in this thread? Elevator pitch -- GPLv3 C code, snapshot-based, compresses, encrypts, deduplicates, can back up clients without installing an agent (just need ssh, bash, tar, and find commands on client for "pull" backups), push backups can have restricted permissions (i.e., give a client permission to push backups only, but not delete backups, or give a user restore-only permissions). Uses tar to collect the data, stores metadata in an SQLite DB on the server, files are stored in LZO format (can be read directly with lzop) (unless client-side encryption is used, but the data can still be decrypted with openssl then decompressed with lzop). Encryption is public-key based instead of needing to keep a shared symmetric key or passphrase laying around on your backup server.

I created and use Snebu -- I'm working on getting it submitted to Fedora (waiting on package review now), doing daily snapshots of my fleet to a raspberry pi with external 12 TB WD Easystore drive. Provides push or pull based backups, granular access permissions, client-side public key encryption (RSA + AES-256) with HMAC validation, server-based data catalog housed in SQLite, multiple client support, global (cross client) file-level deduplication and compression. Works great for backing up a large range of OS versions since the client-side doesn't need an agent -- just bash, tar, find, and ssh.

snebu (c) - https://github.com/derekp7/snebu