zfsnapr
ShellCheck
zfsnapr | ShellCheck | |
---|---|---|
7 | 489 | |
21 | 35,056 | |
- | - | |
5.6 | 8.7 | |
8 months ago | 9 days ago | |
Ruby | Haskell | |
BSD 2-clause "Simplified" License | GNU General Public License v3.0 only |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
zfsnapr
-
Kopia: Open-Source, Fast and Secure Open-Source Backup Software
FreeBSD had a pretty decent option in the base system two decades ago - FFS snapshots and a stock backup tool that would use them automatically with minimal effort, dump(8). Just chuck `-L` at it and your backups are consistent.
Now of course it's all about ZFS, so there's at least snapshots paired with replication - but the story for anything else is still pretty bad, with you having to put all the fiddly pieces together. I'm sure some people taught their backup tool about their special named backup snapshots sprinkled about in `.zfs/snapshot` directories, but given the fiddly nature of it I'm also sure most people just ended up YOLOing raw directories, temporal-smearing be damned.
I know I did!
I finally got around to fixing that last year with zfsnapr[1]. `zfsnapr mount /mnt/backup` and there's a snapshot of the system - all datasets, mounted recursively - ready for whatever backup tool of the year is.
I'm kind of disappointed in mentioning it over on the Practical ZFS forum that the response was not "why didn't you just use ", but "I can see why that might be useful".
Well, yes, it makes backups actually work.
> Also, it's unclear to me what happens if you attempt a snapshot in the middle of something like a database transaction or even a basic file write. Seems likely that the snapshot would still be corrupted
A snapshot is a point-in-time image of the filesystem at a given point. Any ACID database worth the name will roll back the in-flight transaction just like they would if you issued it a `kill -9`.
For other file writes, that's really down to whether or not such interruptions were considered by the writer. You may well have half-written files in your snapshot, with the file contents as they were in between two write() calls. Ideally this will only be in the form of temporary files, prior to their rename() over the data they're replacing.
For everything else - well, you have more than one snapshot backed up, right?
1: https://github.com/Freaky/zfsnapr
-
ZFS for Dummies
I make remote snapshot backups with Borg using this: https://github.com/Freaky/zfsnapr
zfsnapr mounts recursive snapshots on a target directory so you can just point whatever backup tool you like at a normal directory tree.
I still use send/recv for local backups - I think it's good to have a mix of strategies.
-
BorgBackup, Deduplicating archiver with compression and encryption
This is why I made https://github.com/Freaky/zfsnapr
Instead of working out how to teach my backup tools about snapshots, I just mount them in a subtree and use that as a chroot env.
-
Ask HN: Can I see your scripts?
borg-backup.sh, which runs my remote borg backups off a cronjob: https://github.com/Freaky/borg-backup.sh
zfsnapr, a ZFS recursive snapshot mounter - I run borg-backup.sh using this to make consistent backups: https://github.com/Freaky/zfsnapr
mkjail, an automatic minimal FreeBSD chroot environment builder: https://github.com/Freaky/mkjail
run-one, a clone of the Ubuntu scripts of the same name, which provides a slightly friendlier alternative to running commands with flock/lockf: https://github.com/Freaky/run-one
-
Correct Backups Require Filesystem Snapshots
I wrote https://github.com/Freaky/zfsnapr a few months ago so I could finally have point-in-time consistent Borg backups with ZFS snapshots, without having the mess of teaching Borg where every .zfs directory was.
It recursively snapshots mounted pools, and recursively mounts snapshots of the mounted datasets into a target ready to point your backup tools at. I do so via a chroot so I didn't need to make any changes to my Borg setup - just to how I run it.
-
Snapshot stat changes on access
This is the approach I take with zfssnapr - make a recursive snapshot of pools and then use mountpoint/canmount to recursively mount datasets on a location. Then I can just point borg at it without having to teach it where exactly each .zfs directory is.
- zfsnapr — recursively mount a system snapshot on a given location
ShellCheck
-
How I use Devbox in my Elm projects
These projects use Caddy as my local development server, Dart Sass for converting my Sass files to CSS, elm, elm-format, elm-optimize-level-2, elm-review, elm-test (only in Calculator), ShellCheck to find bugs in my shell scripts, and Terser to mangle and compress JavaScript code.
-
Ask HN: Is there a GUI for bash shell?
ncurse, dialog, zenity[2]. i/o buffering may be an issue [3a,3b]
Assuming using same account, use history command to show past commands[0a, 0b]
'load random example' on shellcheck using own custom examples from history command.[1]
--------
[3a] : http://www.gnu.org/software/coreutils/manual/html_node/stdbu...
[3b] : http://unix.stackexchange.com/questions/25372/how-to-turn-of...
[2] : http//funprojects.blog/2021/01/25/zenity-command-line-dialogs/
[1] : http://www.shellcheck.net/
[0a] : http://www.tecmint.com/history-command-examples/
[0b] : http://www.tecmint.com/remember-linux-commands/
web based documentation: https://www.tecmint.com/linux-commands-cheat-sheet/
commands grouped by typical usage patterns : https://www.tecmint.com/essential-linux-commands/
-
DevSecOps with AWS- IaC at scale - Building your own platform - Part 1
... #************************** Terraform ************************************* ARG TERRAFORM_VERSION=1.7.3 RUN set -ex \ && curl -O https://releases.hashicorp.com/terraform/${TERRAFORM_VERSION}/terraform_${TERRAFORM_VERSION}_linux_amd64.zip && unzip terraform_${TERRAFORM_VERSION}_linux_amd64.zip -d /usr/local/bin/ RUN set -ex \ && mkdir -p $HOME/.terraform.d/plugin-cache && echo 'plugin_cache_dir = "$HOME/.terraform.d/plugin-cache"' > ~/.terraformrc #************************* Terragrunt ************************************* ARG TERRAGRUNT_VERSION=0.55.1 RUN set -ex \ && wget https://github.com/gruntwork-io/terragrunt/releases/download/v${TERRAGRUNT_VERSION}/terragrunt_linux_amd64 -q \ && mv terragrunt_linux_amd64 /usr/local/bin/terragrunt \ && chmod +x /usr/local/bin/terragrunt #*********************** Terramate **************************************** ARG TERRAMATE_VERSION=0.4.5 RUN set -ex \ && wget https://github.com/mineiros-io/terramate/releases/download/v${TERRAMATE_VERSION}/terramate_${TERRAMATE_VERSION}_linux_x86_64.tar.gz \ && tar -xzf terramate_${TERRAMATE_VERSION}_linux_x86_64.tar.gz \ && mv terramate /usr/local/bin/terramate \ && chmod +x /usr/local/bin/terramate #*********************** tfsec ******************************************** ARG TFSEC_VERSION=1.28.5 RUN set -ex \ && wget https://github.com/aquasecurity/tfsec/releases/download/v${TFSEC_VERSION}/tfsec-linux-amd64 \ && mv tfsec-linux-amd64 /usr/local/bin/tfsec \ && chmod +x /usr/local/bin/tfsec \ && terragrunt --version #**********************Terraform docs ************************************ ARG TERRRAFORM_DOCS_VERSION=0.17.0 RUN set -ex \ && curl -sSLo ./terraform-docs.tar.gz https://terraform-docs.io/dl/v${TERRRAFORM_DOCS_VERSION}/terraform-docs-v${TERRRAFORM_DOCS_VERSION}-$(uname)-amd64.tar.gz \ && tar -xzf terraform-docs.tar.gz \ && chmod +x terraform-docs \ && mv terraform-docs /usr/local/bin/terraform-docs #********************* ShellCheck ***************************************** ARG SHELLCHECK_VERSION="stable" RUN set -ex \ && wget -qO- "https://github.com/koalaman/shellcheck/releases/download/${SHELLCHECK_VERSION?}/shellcheck-${SHELLCHECK_VERSION?}.linux.x86_64.tar.xz" | tar -xJv \ && cp "shellcheck-${SHELLCHECK_VERSION}/shellcheck" /usr/bin/ \ && shellcheck --version ...
-
Ask HN: Popular open source tool originally written in Haskell?
ShellCheck: https://github.com/koalaman/shellcheck
-
Google ZX – A tool for writing better scripts
If I want to write better shell scripts I usually run shellcheck and adjust accordingly or if I need facilities not provided by the shell i switch to a full fledged programming language. Ans oh yes, `sh` is present almost on every BSD and Linux box for free so I consider it an important thing to at least be comfortable with.
shellcheck: https://www.shellcheck.net/
-
How I use Nix in my Elm projects
When I run nix-shell at the root of the project it puts me in a Nix shell that contains, among other programs, caddy and shellcheck. Notice that in the shellHook I add the project's shell scripts to the PATH. So once I'm in the Nix shell I can, among other things:
- Ask HN: A Bash guide for Posix programmers?
-
Regex support to list modules in .cabal?
I have also seen some projects on github like ShellCheck which first make a library, expose all the modules and then simple add that do build-depends of the final executable. Is this the recommended approach than having just one executable and adding all the modules to other-modules:?
-
Shellcheck finds bugs in your shell scripts
The error checks can be pretty arcane:
https://github.com/koalaman/shellcheck/wiki/Checks
-
Is there a syntax checker?
Similar to for instance shellcheck to check the syntax of shell scripts, is there an equivalent for the set of roff commands typically used in a (Linux) man page? I'm aware that e.g. pandoc permits the conversion of an other format (e.g., org) to both roff man and roff ms.
What are some alternatives?
BorgBackup - Deduplicating archiver with compression and authenticated encryption.
bash-language-server - A language server for Bash
ioztat - ioztat is a storage load analysis tool for OpenZFS. It provides iostat-like statistics at an individual dataset/zvol level.
shellharden - The corrective bash syntax highlighter
benchmarks - Benchmarks of different backup tools.
shfmt - Dockernized shfmt. This formats shell script.
RcloneZFSBackup - Backup ZFS snapshots to cloud storage using RCLone
shfmt - A shell formatter (sh/bash/mksh)
borgmatic - Simple, configuration-driven backup software for servers and workstations
PowerShell - PowerShell for every system!
borgtui - A nice TUI for BorgBackup
efm-langserver - General purpose Language Server