microk8s VS kops

If you want hands-on practice you should have a running Kubernetes cluster (I used MicroK8s for this tutorial) and Helm (see how to install on Installing Helm tutorial). It is important that you understand the basics of these tools to fully understand.

And install microk8s:

MicroK8s is a lightweight, batteries included Kubernetes distribution by Canonical designed for running edge workloads which also happens to be developer-friendly and a great choice for building your own homelab. The following lab covers how to install and run MicroK8s on your own edge node running Ubuntu 22.04 LTS, deploy the NGINX web service and exposing your NGINX website to the Internet with SSL/TLS enabled using AWS resources included within the Free Tier.

One quick and easy win I can recommend, is microk8s.

I had the same problem (and there's a github issue about this: https://github.com/canonical/microk8s/issues/2186). I swapped to k3s and the usage was half of what microk8s used.

microk8s currently has a showstopping issue that makes it guaranteed to have an irrecoverable failure in HA mode. see https://github.com/canonical/microk8s/issues/3227

k0s is better but also has a lot of bugs. it's the closest to vanilla kubernetes among all the distributions.

> like the simplest GPU support

linux users should be ready to install the nvidia device plugin. if they can't do that, they're never going to succeed in running a gpu accelerated application on their cluster anyway.

> like bootstrapping

in my experience, writing all the bootstrap scripts is painful. but now that there's chatgpt, so much of the drudgery as gone away.

MicroK8S to spin up a Kubernetes cluster

They now power my microk8s/x86 cluster (in addition to my 8-node Raspberry Pi4 ARM64 microk8s cluster), microceph cluster and my LXD cluster, and all are configured with WOL, so I can bring up the cluster from any machine in the homelab, on demand.

The general problem of patching resource definitions that are not fully under your control has also been recognized for some time. This is true of default resources created and updated by cluster maintenance tools (e.g. kOps), or by public helm charts that you use to install common services and operators (e.g. nginx-ingress, cert-manager, and so on). High quality charts will allow you to override the configuration of important components such as service account references, but some simpler charts offer much less configuration.

The kops documentation is not very clear about this.

Very fast progress and great article with lots of useful information. I found myself nodding a lot having recently seen GitLab Dedicated https://docs.gitlab.com/ee/subscriptions/gitlab_dedicated/ being developed that had my similar challenges. I wonder what other people think of Kops https://kops.sigs.k8s.io/

note: I work with kube on a daily basis. All current clusters under my care use a managed backplane service though there are a number of small (3+ years-old) clusters that I still keep an eye on that were initialized using kops.

kOps with your own instances.

Companies run their own clusters (sometimes for cost reasons), using tools like kops, and kubeadm to set up their own clusters.

curl -Lo kops https://github.com/kubernetes/kops/releases/download/$(curl -s https://api.github.com/repos/kubernetes/kops/releases/latest | grep tag_name | cut -d '"' -f 4)/kops-linux-amd64 chmod +x kops sudo mv kops /usr/local/bin/

Kops is a cluster setup and management command line tool that deploys a Kubernetes cluster to AWS. It provides configuration abstractions such as manifest YAML files that facilitate node and components configuration. And like Ansible, it will provide dry-run capabilities and ensures idempotency of changing the nodes.