k3s VS k3s

In this demo, we will be using k3d a lightweight wrapper to run k3s (Rancher Lab’s minimal Kubernetes distribution) in docker, to create 4 separate clusters named control-plane, cluster-1, cluster-2, and cluster-3 respectively.

To run the demo locally, we recommend k3d a lightweight wrapper to run k3s (Rancher Lab’s minimal Kubernetes distribution) in docker.

I've been messing around with k8s (k3s) lately, and got to the "issue" of downtime/inconsistencies caused by one of multiple workers being down, who had pods running on them. I found a couple useful parameters here that helped me reduce the time needed to redeploy the old pods on other nodes, as well as stop sending requests to the NotReady node. But that got me thinking, how long should k8s wait before doing these things? Or is there perhaps a better option for increasing avaliability?

Local Kubernetes clusters are clusters that are running on the individual computer of the developer. There are many tools that provide such an environment, such as Minikube, microk8s, k3s, or kind. While they are not all the same, their use as a development environment is quite comparable.

Since the developer is the only one who has to access this cluster for development, local clusters can be a feasible solution for this purpose. Over time, several solutions have emerged that are particularly made for running Kubernetes in local environments. The most important ones are Kubernetes in Docker (kind), MicroK8s, minikube and k3s. For a comparison of these local Kubernetes options, you can look at this post.

With local Kubernetes environments such as minikube or k3s, developers can create their own Kubernetes clusters on their local computers. This often leads to developers struggling with the management and setup of these pared-down Kubernetes technologies that are also not completely realistic compared to “real-world”, cloud-based environments. The upside of this approach is that the developers have full control over their environment and can independently create it whenever they need it.

K3d - a lightweight wrapper to run k3s in docker.

Opinionated meaning it picks, install, patches your CNI/Ingress/Load Balancer/DNS Server/Metrics Server/Monitoring Setup.

k3s is probably most well known as it ships with bunch of preinstall software: https://github.com/k3s-io/k3s so you can just start throwing yaml files at cluster and handling workloads. It's what I use for my homelab.

Paid things I've heard of include OpenStack and SideroLabs. Haven't used personally by SRE coworkers say good things about them.

For self-hosting I've found https://k3s.io to be really good from the SUSE people. Works on basically any Linux distro and makes self-hosting k8s not miserable.

Yes it’s going to depend on which k8s distribution you’re using. We have work in-progress for k3s to natively support nix-snapshotter: https://github.com/k3s-io/k3s/pull/9319

For other distributions, nix-snapshotter works with official containerd releases so it’s just a matter of toml configuration and a systemd unit for nix-snapshotter.

We run Kubernetes outside of NixOS, but yes the NixOS modules provided by the nix-snapshotter certainly make it simple.

K3S: is a lightweight distribution of Kubernetes that is designed for resource-constrained environments. It is an excellent option for running Kubernetes on a virtual machine or cloud server.

I recently purchased a used Lenovo M900 Think Centre (i7 with 32GB RAM) from eBay to expand my mini-homelab, which was just a single Synology DS218+ plugged into my ISP's router (yuck!). Since I've been spending a big chunk of time at work playing around with Kubernetes, I figured that I'd put my skills to the test and run a k3s node on the new server. While I was familiar with k3s before starting this project, I'd never actually run it before, opting for tools like kind (and minikube before that) to run small test clusters for my local development work.

Kubernetes - k3s distribution

I’ve created a local cluster with K3S and installing Windmill could not be simpler with just one chart to configure, which already has sane defaults to get started. For this demo we will also configure workers to passthrough environment variables to our scripts so that they have access to the Kubernetes API server for later.

You should be familiar with Kubernetes and have set up a Kubernetes cluster. I recommend k3s.