kube-monkey VS k3s

Not in the typical sense, but we have plenty of standard practices and cross-team checkpoints to limit risk. By the time we're deploying changes to production, the work has had a card created, assigned points (which necessarily involves discussing scope and risk), architected (as a group), code peer reviewed, hit unit tests (automated), integration tests (automated), functional tests (automated), smoke tested (automated) end-to-end tests (a few automated, but mostly manual by QA), acceptance tested (by QA and business), resilience tests (chaos engineering with kube-monkey), been deployed to at least 3 environments (with the same exact same artifacts, just with config changes), and monitored for failures (pod restarts, log anomalies, etc -- all automated). Deploy to production is well communicated, and ANY team can request a halt to the deploy if they have concerns.

Part of being a cloud native company means designing services for failure. What happens, for example, if the payment service/pod goes down? Do the rest of your services continue operating normally? One thing tools like kube-monkey does is automatically kill pods for you on a certain date at a certain time in order to plan for failure events. Just wondering if anyone has dove into the deep end with this type of tooling and really just gone all out, besides Netflix?

Chaos Mesh is a popular solution (about 5k GitHub stars), but — obviously — not the only one. E.g., Litmus is a powerful platform to test many things, and kube-monkey might be a good option for more basic stuff.

Exactly the kind of thing I love to see! Sounds like a great use case for a tool like kube-monkey as well.

Kube Monkey

For self-hosting I've found https://k3s.io to be really good from the SUSE people. Works on basically any Linux distro and makes self-hosting k8s not miserable.

Yes it’s going to depend on which k8s distribution you’re using. We have work in-progress for k3s to natively support nix-snapshotter: https://github.com/k3s-io/k3s/pull/9319

For other distributions, nix-snapshotter works with official containerd releases so it’s just a matter of toml configuration and a systemd unit for nix-snapshotter.

We run Kubernetes outside of NixOS, but yes the NixOS modules provided by the nix-snapshotter certainly make it simple.

K3S: is a lightweight distribution of Kubernetes that is designed for resource-constrained environments. It is an excellent option for running Kubernetes on a virtual machine or cloud server.

I recently purchased a used Lenovo M900 Think Centre (i7 with 32GB RAM) from eBay to expand my mini-homelab, which was just a single Synology DS218+ plugged into my ISP's router (yuck!). Since I've been spending a big chunk of time at work playing around with Kubernetes, I figured that I'd put my skills to the test and run a k3s node on the new server. While I was familiar with k3s before starting this project, I'd never actually run it before, opting for tools like kind (and minikube before that) to run small test clusters for my local development work.

Kubernetes - k3s distribution

I’ve created a local cluster with K3S and installing Windmill could not be simpler with just one chart to configure, which already has sane defaults to get started. For this demo we will also configure workers to passthrough environment variables to our scripts so that they have access to the Kubernetes API server for later.

You should be familiar with Kubernetes and have set up a Kubernetes cluster. I recommend k3s.

K3s' go.mod[0] is insane.

[0] https://github.com/k3s-io/k3s/blob/master/go.mod