hashi-ui VS k3s

Nomad also scales really well. In my experience swarm had a lot of issues with going above 10 machines in a cluster. Stuck containers, containers that are there but swarm can't see them and more. But still i loved using swarm with my 5 node arm cluster, it is a good place to start when you hit the limit of a single node.

> The only serious downsides is having to use the HCL DSL ( https://github.com/hashicorp/hcl ) and their web UI being read only in the last versions that i checked.

1. IIRC you can run jobs directly from UI now, but IMO this is kinda useless. Running a job is simple as 'nomad run jobspec.nomad'. You can also run a great alternative UI ( https://github.com/jippi/hashi-ui ).

2. IMO HCL > YAML for job definitions. I've used both extensively and HCL always felt much more human friendly. The way K8s uses YAML looks to me like stretching it to it's limits and barely readable at times with templates.

One thing that makes nomad a go-to for me is that it is able to run workloads pretty much anywhere. Linux, Windows, FreeBSD, OpenBSD, Illumos and ofc Mac.

Hashicorp Nomad combined with Hashi-ui (https://github.com/jippi/hashi-ui) comes relatively close, but is disqualified because it provides no support for easy to use provisioning. Azkaban comes relatively close, but seems not to have strong supoort for containers.

For self-hosting I've found https://k3s.io to be really good from the SUSE people. Works on basically any Linux distro and makes self-hosting k8s not miserable.

Yes it’s going to depend on which k8s distribution you’re using. We have work in-progress for k3s to natively support nix-snapshotter: https://github.com/k3s-io/k3s/pull/9319

For other distributions, nix-snapshotter works with official containerd releases so it’s just a matter of toml configuration and a systemd unit for nix-snapshotter.

We run Kubernetes outside of NixOS, but yes the NixOS modules provided by the nix-snapshotter certainly make it simple.

K3S: is a lightweight distribution of Kubernetes that is designed for resource-constrained environments. It is an excellent option for running Kubernetes on a virtual machine or cloud server.

I recently purchased a used Lenovo M900 Think Centre (i7 with 32GB RAM) from eBay to expand my mini-homelab, which was just a single Synology DS218+ plugged into my ISP's router (yuck!). Since I've been spending a big chunk of time at work playing around with Kubernetes, I figured that I'd put my skills to the test and run a k3s node on the new server. While I was familiar with k3s before starting this project, I'd never actually run it before, opting for tools like kind (and minikube before that) to run small test clusters for my local development work.

Kubernetes - k3s distribution

I’ve created a local cluster with K3S and installing Windmill could not be simpler with just one chart to configure, which already has sane defaults to get started. For this demo we will also configure workers to passthrough environment variables to our scripts so that they have access to the Kubernetes API server for later.

You should be familiar with Kubernetes and have set up a Kubernetes cluster. I recommend k3s.

K3s' go.mod[0] is insane.

[0] https://github.com/k3s-io/k3s/blob/master/go.mod