kind VS tilt-extensions

To get started, you'll need to install clusteradm and kubectl and start up three Kubernetes clusters. To simplify cluster administration, this article starts up three kind clusters with the following names and purposes:

Kind: is a tool for running local Kubernetes clusters using Docker container "nodes." It was primarily designed for testing Kubernetes itself but can also be used for local development or continuous integration.

Fortunately, just as projects like kind and Minikube enable developers to spin up a local Kubernetes environment in no time, CRC, also known as OpenShift Local and a recursive acronym for "CRC - Runs Containers", offers developers a local OpenShift environment by means of a pre-configured VM similar to how Minikube works under the hood.

I recently purchased a used Lenovo M900 Think Centre (i7 with 32GB RAM) from eBay to expand my mini-homelab, which was just a single Synology DS218+ plugged into my ISP's router (yuck!). Since I've been spending a big chunk of time at work playing around with Kubernetes, I figured that I'd put my skills to the test and run a k3s node on the new server. While I was familiar with k3s before starting this project, I'd never actually run it before, opting for tools like kind (and minikube before that) to run small test clusters for my local development work.

Features compared to https://kind.sigs.k8s.io/

Kind allows you to run a Kubernetes cluster inside Docker. This is incredibly useful for developing Helm charts, Operators, or even just testing out different k8s features in a safe way.

If you're just messing around, just use kind (https://kind.sigs.k8s.io) or minikube if you want VMs (https://minikube.sigs.k8s.io). Both work on ARM-based platforms.

You can also use k3s; it's hella easy to get started with and it works great.

We'll install APIClarity into a Kubernetes cluster to test our API documentation. We're using a Kind cluster for demonstration purposes. Of course, if you have another Kubernetes cluster up and running elsewhere, all steps also work there.

yes I know there is lens, but it does not allow me to see logs of multiple pods at same time and what is even more important it is not friendly for ephemeral clusters - in my case with help of kind I am recreating whole cluster each time from scratch

Unit tests were written against an in-memory Kubernetes API server using the controller-runtime/pkg/envtest library. Envtest allowed us to iterate quickly since we could run tests against a fresh API cluster that started up in around 5 seconds instead of having to spin up a new cluster every time we wanted to run a test suite. Even existing micro-cluster tools like Kind could not get us that level of performance. Since envtest is also not packaged with any controllers, we could also set our test cluster to a specific state and be sure that this state would not be modified unless we explicitly did so in our test code. This allowed us to fully test specific edge-cases without having to worry about control plane-level controllers modifying various objects out from underneath us.

The first is the load function which loads Tilt extensions. It's a way to expand the tool's features, and several are available. Here we are using docker_build_with_restart, which will update the container running inside our Kubernetes cluster.

This is the central viewport into manual resource control and environment enhancement through the open-source extensions for Tilt.

argocd for cd Tilt

Recently, I found Tilt [0] to be a good partner of mine to run "all services locally". It can be compared to "webpack (live-reloading, a lot of configuration possibilities) for backend". You want to run a bunch of services directly? Use local_resource()/local(). You have Procfile? There is procfile() function. You have docker-compose.yml with databases? You can run it too with docker_compose(). You want have Tiltfile and include them all-together? There is load(). You need some web-ui for frontend devs and a nice log browser? It is there too. You need to do some extra steps before running a service? You want to update your local cluster with newly built image on file save? No problem, tilt will do that with k8s_yaml() function. Tilt uses Titlfiles for configuration, which are written Pythonish Starlark language and you use them to run any specific logic there.

Also, I am not very lucky in having resemble 1:1 k8s cluster locally. You could be close but as long as you don't run already in cloud you will have different configuration (additional annotations, various quirks that do not exist in kind/k3s but they are on GCP). However, making dedicated dev environments in the cloud might be very costly and incur a lot of additional tinkering.

[0]: https://tilt.dev/

The article doesn't mention k3d (https://k3d.io/) which is a variant of k3s that runs in docker (rather than a VM) - very nice for k8s dev/test on developer workstations.

It integrates very nicely with https://tilt.dev/ also (another very useful tool for k8s related dev/test).

I'd add Tilt

Tilt

Apart from Skaffold, you should also take a look at telepresence, okteto.com and tilt.dev

For sources stored on the Windows filesystem, any changes made by Windows applications such as Visual Studio do not trigger any file change notifications as far as Linux apps are concerned. This means that all "live rebuild"-type tools don't work (examples: webpack --watch, jekyll --interactive, and Tilt.dev) when running under WSL2. This unfortunately renders many modern dev workflows unviable.