zfsnapr VS dotfiles

FreeBSD had a pretty decent option in the base system two decades ago - FFS snapshots and a stock backup tool that would use them automatically with minimal effort, dump(8). Just chuck `-L` at it and your backups are consistent.

Now of course it's all about ZFS, so there's at least snapshots paired with replication - but the story for anything else is still pretty bad, with you having to put all the fiddly pieces together. I'm sure some people taught their backup tool about their special named backup snapshots sprinkled about in `.zfs/snapshot` directories, but given the fiddly nature of it I'm also sure most people just ended up YOLOing raw directories, temporal-smearing be damned.

I know I did!

I finally got around to fixing that last year with zfsnapr[1]. `zfsnapr mount /mnt/backup` and there's a snapshot of the system - all datasets, mounted recursively - ready for whatever backup tool of the year is.

I'm kind of disappointed in mentioning it over on the Practical ZFS forum that the response was not "why didn't you just use ", but "I can see why that might be useful".

Well, yes, it makes backups actually work.

> Also, it's unclear to me what happens if you attempt a snapshot in the middle of something like a database transaction or even a basic file write. Seems likely that the snapshot would still be corrupted

A snapshot is a point-in-time image of the filesystem at a given point. Any ACID database worth the name will roll back the in-flight transaction just like they would if you issued it a `kill -9`.

For other file writes, that's really down to whether or not such interruptions were considered by the writer. You may well have half-written files in your snapshot, with the file contents as they were in between two write() calls. Ideally this will only be in the form of temporary files, prior to their rename() over the data they're replacing.

For everything else - well, you have more than one snapshot backed up, right?

1: https://github.com/Freaky/zfsnapr

I make remote snapshot backups with Borg using this: https://github.com/Freaky/zfsnapr

zfsnapr mounts recursive snapshots on a target directory so you can just point whatever backup tool you like at a normal directory tree.

I still use send/recv for local backups - I think it's good to have a mix of strategies.

This is why I made https://github.com/Freaky/zfsnapr

Instead of working out how to teach my backup tools about snapshots, I just mount them in a subtree and use that as a chroot env.

borg-backup.sh, which runs my remote borg backups off a cronjob: https://github.com/Freaky/borg-backup.sh

zfsnapr, a ZFS recursive snapshot mounter - I run borg-backup.sh using this to make consistent backups: https://github.com/Freaky/zfsnapr

mkjail, an automatic minimal FreeBSD chroot environment builder: https://github.com/Freaky/mkjail

run-one, a clone of the Ubuntu scripts of the same name, which provides a slightly friendlier alternative to running commands with flock/lockf: https://github.com/Freaky/run-one

I wrote https://github.com/Freaky/zfsnapr a few months ago so I could finally have point-in-time consistent Borg backups with ZFS snapshots, without having the mess of teaching Borg where every .zfs directory was.

It recursively snapshots mounted pools, and recursively mounts snapshots of the mounted datasets into a target ready to point your backup tools at. I do so via a chroot so I didn't need to make any changes to my Borg setup - just to how I run it.

This is the approach I take with zfssnapr - make a recursive snapshot of pools and then use mountpoint/canmount to recursively mount datasets on a location. Then I can just point borg at it without having to teach it where exactly each .zfs directory is.

True, but you can represent that in the db as a a CLI invocation to run in a subshell.

The big gain from something like carapace or my theoretical SQLite-based completion system is faster startup time. I had to remove zsh-completions from my shell setup as it added too much to the startup time (https://github.com/bbkane/dotfiles/blob/master/zsh/README_no...)

"small embeddable dynamic languages" are usually used to configure or program other larger compiled applications. This is bes understood by example:

https://create.roblox.com/docs/tutorials/scripting/basic-scr... - make a mini game in Roblox

https://github.com/openresty/lua-nginx-module?tab=readme-ov-... - configure and extend NGINX

https://wezfurlong.org/wezterm/config/lua/general.html - make your terminal more useful (my personal config changes the tab color based on the process name - https://github.com/bbkane/dotfiles/blob/master/wezterm/dot-c...

https://m.youtube.com/watch?v=MQBr9hwf0BY - configure your text editor

I switched to iTerm2 a few years ago due to blurry fonts on zoom with Terminal.app . Wonder if that's still a problem?

A few months ago I switched to WezTerm and, after some config wrestling, I've been very happy using it (https://github.com/bbkane/dotfiles/tree/master/wezterm).

Yes, but it's super awkward to actually use day to day

I've got something of a wrapper script at https://github.com/bbkane/dotfiles/blob/8573e44d0f9fb5ddcbdc...

It doesn't read from files unfortunately, but https://openobserve.ai/ is very easy to set up locally (single binary) and send otel logs/metrics/traces to.

Here's how I run it locally for my little shovel project - https://github.com/bbkane/shovel#run-the-webapp-locally-with... .

Also linked from that README is an Ansible playbook to start OpenObserve as a systems service on a Linux VM.

Alternatively, see the shovel codebase I linked above for a "stdout" TracerProvider. You could do something like that to save to a file, and then use a tool to prettify the JSON. I have a small script to format json logs at https://github.com/bbkane/dotfiles/blob/2df9af5a9bbb40f2e101...

That sounds super similar my setup ( https://github.com/bbkane/dotfiles/tree/master/zsh ). I'll check out a few of those I haven't yet.

Here's a small script I use often to tag commits with Git.

It shows the current status, lists out the most recent tags, prompts for a new tab and message, and finally pushes.

Everything is colorized so it's easy to read and I use it quite often for Golang projects.

https://github.com/bbkane/dotfiles/blob/e30c12c11a61ccc758f7...

Ooh I'm doing this too, but with Python to add a "category" field (based mostly on description), nushell to munge the CSV into more CSVs so I can build html charts and tables with this script. in my opinion, transforming the two original CSVs (checking account and credit card history) into the html doc with all the charts is best done as this sort of pipeline so you can replace bits as you find better alternatives (for example I started with SQLite instead of nushell for the "child CSV" parts)

I use https://github.com/bbkane/dotfiles/blob/master/bin_common/bin_common/git_lines_changed_tsv.sh to turn this into a tsv which can then be charted by piping to https://github.com/bbkane/dotfiles/blob/master/bin_common/bin_common/scatterplot.py .