microk8s VS docker

And install microk8s:

MicroK8s is a lightweight, batteries included Kubernetes distribution by Canonical designed for running edge workloads which also happens to be developer-friendly and a great choice for building your own homelab. The following lab covers how to install and run MicroK8s on your own edge node running Ubuntu 22.04 LTS, deploy the NGINX web service and exposing your NGINX website to the Internet with SSL/TLS enabled using AWS resources included within the Free Tier.

One quick and easy win I can recommend, is microk8s.

I had the same problem (and there's a github issue about this: https://github.com/canonical/microk8s/issues/2186). I swapped to k3s and the usage was half of what microk8s used.

microk8s currently has a showstopping issue that makes it guaranteed to have an irrecoverable failure in HA mode. see https://github.com/canonical/microk8s/issues/3227

k0s is better but also has a lot of bugs. it's the closest to vanilla kubernetes among all the distributions.

> like the simplest GPU support

linux users should be ready to install the nvidia device plugin. if they can't do that, they're never going to succeed in running a gpu accelerated application on their cluster anyway.

> like bootstrapping

in my experience, writing all the bootstrap scripts is painful. but now that there's chatgpt, so much of the drudgery as gone away.

MicroK8S to spin up a Kubernetes cluster

They now power my microk8s/x86 cluster (in addition to my 8-node Raspberry Pi4 ARM64 microk8s cluster), microceph cluster and my LXD cluster, and all are configured with WOL, so I can bring up the cluster from any machine in the homelab, on demand.

We will be using docker and microk8s from Canonical. For running our software during development, we will be using skaffold which is a great tool developed by Google.

Everything is working fine, but it is not currently doing the automatic code reload. I tried mounting the volume but there is some conflict with the way docker and virtualbox handles files such that the conflict leads to file changes from the host not reflected in docker container. (That's not the first link I have that appears related to this problem, it's just the first I found while googling it on another day)...

github.com/docker/docker/client

Moby is the core of Docker. https://github.com/docker/docker redirects to it.

Docker however has made some major screwups in their IPv6 implementation, so it’s not so trivial as you’d expect it to be. There’s a whole discussion about it here: https://github.com/docker/docker.github.io/issues/6075 but the gist is: Docker cannot auto-assign addresses (SLAAC) like a normal endpoint, it also cannot request a prefix (DHCPv6 PD) like a router, you have to set it all up manually.

[1] https://github.com/docker/docker.github.io/issues/5325

That said, Swarm is definitely in maintenance mode, which is actually how some people like it, given that this means pretty good stability for the current features, instead of breaking changes, which are a slightly larger concern in the case of Kubernetes. However, this also means that new features will get implemented only occasionally, if ever - the last big one that i recall was Swarm jobs. Essentially, i'd say that it's still just a question of time until Swarm dies, but for now it's in the sweet spot of stability and usability if all that you need is a pretty simple and lightweight orchestrator.

From scrounging through the internet, my understanding is if multiple docker containers are run based on the same image, the only extra disk space used is what the writable layer uses and the read-only image data is shared by all the containers. https://github.com/docker/docker.github.io/issues/1520#issuecomment-305179362

Wowww: https://github.com/docker/docker.github.io/issues/6910#issue...

What the hell does "Docker's new direction to go back to its roots and focus on developer tooling" even mean? Docker is developer tooling, that's the whole product. That's not "going back to your roots" that's just doing your damn job.