k3d VS microk8s

K3D: is a lightweight distribution of Kubernetes designed for resource-constrained environments. It is an excellent option for running Kubernetes on virtual machines or cloud servers.

k3d is a lightweight wrapper that makes running Kubernetes (specifically, the lightweight k3s distribution) in Docker straightforward and efficient. It's designed to provide developers with a quick and easy way to test Kubernetes without the overhead of setting up a full cluster.

If you customer prefers to run the standard docker engine you could use k3d

You don’t need to run Rancher from a Kubernetes cluster, the rancher/rancher image works fine with Docker (it uses k3d, aka « k3s in docker » : https://k3d.io/).

K3d and Skaffold for local development

If you are a developer and want to learn how to deploy applications to a cluster, getting a cluster up an running can be a daunting task in it's own rights. There are many ways to do it: spinning up local virtual machines and configuring from scratch or using tools like minikube, etc. You may not care for the pain of setting up and configuring a cluster, and if that is you, then the quickest way that I have found is using k3d.

And install microk8s:

MicroK8s is a lightweight, batteries included Kubernetes distribution by Canonical designed for running edge workloads which also happens to be developer-friendly and a great choice for building your own homelab. The following lab covers how to install and run MicroK8s on your own edge node running Ubuntu 22.04 LTS, deploy the NGINX web service and exposing your NGINX website to the Internet with SSL/TLS enabled using AWS resources included within the Free Tier.

One quick and easy win I can recommend, is microk8s.

I had the same problem (and there's a github issue about this: https://github.com/canonical/microk8s/issues/2186). I swapped to k3s and the usage was half of what microk8s used.

microk8s currently has a showstopping issue that makes it guaranteed to have an irrecoverable failure in HA mode. see https://github.com/canonical/microk8s/issues/3227

k0s is better but also has a lot of bugs. it's the closest to vanilla kubernetes among all the distributions.

> like the simplest GPU support

linux users should be ready to install the nvidia device plugin. if they can't do that, they're never going to succeed in running a gpu accelerated application on their cluster anyway.

> like bootstrapping

in my experience, writing all the bootstrap scripts is painful. but now that there's chatgpt, so much of the drudgery as gone away.

MicroK8S to spin up a Kubernetes cluster

They now power my microk8s/x86 cluster (in addition to my 8-node Raspberry Pi4 ARM64 microk8s cluster), microceph cluster and my LXD cluster, and all are configured with WOL, so I can bring up the cluster from any machine in the homelab, on demand.

We will be using docker and microk8s from Canonical. For running our software during development, we will be using skaffold which is a great tool developed by Google.