zfsbootmenu VS k3s

We use a couple nice home-grown functions in ZFSBootMenu to help debug things. We have a zdebug logging function that's peppered liberally throughout the code base - https://github.com/zbm-dev/zfsbootmenu/blob/master/zfsbootme...

Hitting ctrl-t on our main menu will, when booting with debug logging enabled, show a screen like this: https://imgur.com/Ge75zkP

We also have a flamegraph profiling mechanism that can be enabled with https://github.com/zbm-dev/zfsbootmenu/blob/master/zfsbootme... . That will dump data to a serial port, which when re-assembled, can be used to produce a graph like https://raw.githubusercontent.com/zbm-dev/zfsbootmenu/master...

Bash is suprisingly flexible.

A lot of what's in the Pure Bash Bible is horrifically slow. Many of those things are substantially faster, even when paying the cost of starting a new process, when you use an external and commonly available tool. I wrote a bash performance profiler that outputs data in a format that flamegraph.pl recognizes - it really helped identify where we could improve the performance of ZFSBootMenu.

https://github.com/zbm-dev/zfsbootmenu/releases/tag/v1.12.0

Don't fall in the trap of thinking things have to be written entirely in bash; it's okay to use other tools to help fill in the gaps.

If instead of "opensuse" you're asking for bootloader as grub can't boot from zfs, then, like i metnioned, i don't use grub2, i uninstalled it, instead i'm using https://github.com/zbm-dev/zfsbootmenu

I thought the following was supposed to fix this issue: https://github.com/zbm-dev/zfsbootmenu/commit/84da18e64ebcc0c483e7b2c7d3972f7d91784e63

All release assets, including EFI executables and kernel/initramfs pairs, are signed with signify, which provides a simple method for verifying that the contents of the file are as this project intended. Once you've installed signify (that's left as an exercise, although Void Linux provides the signify package for this purpose), just download the desired assets from the ZFSBootMenu release page, download the file sha256.sig alongside it, and run:

I think the consensus is that you probably shouldn't be installing a ZFS on root using the native installer anymore. They aren't really maintaining the packages that make that work. Instead the suggestion is to go the zfsbootmenu route of installing.

Glad to hear that everything is working for you! I've opened a PR that adds a warning about this condition - it should likely make it into 2.2.0.

You can install following instructions at https://openzfs.github.io/openzfs-docs/Getting%20Started/Debian/Debian%20Bullseye%20Root%20on%20ZFS.html which I've automated with https://github.com/HankB/Linux_ZFS_Root/tree/master/Debian. For scripting, you should also look at https://github.com/zbm-dev/zfsbootmenu. I'd probably go that way if I were starting from scratch.

ZBM provides an amazingly useful script in it's wiki here. This runs when a new kernel is updated by xbps and it snapshots your system before the kernel is installed. This creates a boot environment, and via the magic of ZFS boot environments, allows you to rollback any kernel update to a known, working configuration.

* https://docs.oracle.com/cd/E86824_01/html/E54764/beadm-1m.ht...

> A ZFS boot environment is a bootable clone of the datasets needed to boot the operating system. Creating a BE before performing an upgrade provides a low-cost safeguard: if there is a problem with the update, the system can be rebooted back to the point in time before the upgrade.

* https://klarasystems.com/articles/managing-boot-environments...

Or perhaps:

> In essence, ZFSBootMenu is a small, self-contained Linux system that knows how to find other Linux kernels and initramfs images within ZFS filesystems. When a suitable kernel and initramfs are identified (either through an automatic process or direct user selection), ZFSBootMenu launches that kernel using the kexec command.

* https://github.com/zbm-dev/zfsbootmenu

Opinionated meaning it picks, install, patches your CNI/Ingress/Load Balancer/DNS Server/Metrics Server/Monitoring Setup.

k3s is probably most well known as it ships with bunch of preinstall software: https://github.com/k3s-io/k3s so you can just start throwing yaml files at cluster and handling workloads. It's what I use for my homelab.

Paid things I've heard of include OpenStack and SideroLabs. Haven't used personally by SRE coworkers say good things about them.

For self-hosting I've found https://k3s.io to be really good from the SUSE people. Works on basically any Linux distro and makes self-hosting k8s not miserable.

Yes it’s going to depend on which k8s distribution you’re using. We have work in-progress for k3s to natively support nix-snapshotter: https://github.com/k3s-io/k3s/pull/9319

For other distributions, nix-snapshotter works with official containerd releases so it’s just a matter of toml configuration and a systemd unit for nix-snapshotter.

We run Kubernetes outside of NixOS, but yes the NixOS modules provided by the nix-snapshotter certainly make it simple.

K3S: is a lightweight distribution of Kubernetes that is designed for resource-constrained environments. It is an excellent option for running Kubernetes on a virtual machine or cloud server.

I recently purchased a used Lenovo M900 Think Centre (i7 with 32GB RAM) from eBay to expand my mini-homelab, which was just a single Synology DS218+ plugged into my ISP's router (yuck!). Since I've been spending a big chunk of time at work playing around with Kubernetes, I figured that I'd put my skills to the test and run a k3s node on the new server. While I was familiar with k3s before starting this project, I'd never actually run it before, opting for tools like kind (and minikube before that) to run small test clusters for my local development work.

Kubernetes - k3s distribution

I’ve created a local cluster with K3S and installing Windmill could not be simpler with just one chart to configure, which already has sane defaults to get started. For this demo we will also configure workers to passthrough environment variables to our scripts so that they have access to the Kubernetes API server for later.

You should be familiar with Kubernetes and have set up a Kubernetes cluster. I recommend k3s.