k3s VS kubespray

A lightweight k3s Kubernetes cluster using k3d

Setting up the Kubernetes cluster and the AWS SQS queue is outside the scope of this article, but you can deploy an Amazon EKS cluster by following this guide, or use k3s as a lightweight, local alternative. For setting up an SQS queue, refer to this tutorial.

Start with small-scale experiments using k3s/k3d

This will initialize K3S (a lightweight Kubernetes distribution), and will configure the Kairos SpinKube bundle we will install in a moment.

Finally the main reason for us is the choice of runtime. We have very decent container runtimes (RKE, RHOS, K3s) available to deploy applications. We are very familiar with them, and they offer us a lot of functionality. These all support containers primarily.

I used K3S in my cluster because it is a lightweight, stripped-down version of Kubernetes that’s ideal for running on resource-constrained devices like Raspberry PIs I plan to use in my home lab cluster. K3S can be installed through a shell script:

In the last few years, we’ve witnessed the introduction of several new lightweight Kubernetes distributions. SUSE’s Rancher Labs k3s project was one of the earliest. Canonical now includes Microk8s in Ubuntu. And k0s is a single-binary Kubernetes distribution.

K3S is a Kubernetes distribution made by Rancher, made to be as lightweight as possible while being compatible with Kubernetes production standards.

Jeff led me to K3s using Ansible, a lightweight Kubernetes distribution that is perfect for my home cluster and a pre-defined way of installing it because I don't have pre-requirements nor the idea on how to set it up otherwise.

So now that I've got my hosts deployed, it's time to set up Kubernetes. I've cloned the Kubespray GitHub repo and now I'll run the cluster.yml playbook:

kube-master1:~# sudo apt install python3-pip -y sudo apt install python3-virtualenv -y git clone https://github.com/kubernetes-sigs/kubespray.git VENVDIR=kubespray-venv KUBESPRAYDIR=kubespray ANSIBLE_VERSION=2.12 virtualenv --python=$(which python3) $VENVDIR source $VENVDIR/bin/activate (kubespray-venv) kube-master1:~# cd $KUBESPRAYDIR pip install -U -r requirements.txt cp -rfp inventory/sample inventory/mycluster

For further troubleshooting on any encountered issue, please refer to the official Kubespray repository for comprehensive troubleshooting steps.

When it came to deployment, I had several options, and I chose the hard way: deploying Kubernetes on bare metal nodes using KubeSpray. Troubleshooting bare metal Kubernetes deployments honed my skills in pinpointing issues. This hands-on experience provided a deep understanding of how each component, like the Control Plane, kubelet, Container Runtime, and scheduler, interacts to orchestrate containers.

Worth noting that if you like ansible, Kubespray has had documented air-gap installation since 2018 https://github.com/kubernetes-sigs/kubespray/commit/963c3479...

calico v3.7.3 kubernetes v1.16.0 installed via kubespray https://github.com/kubernetes-sigs/kubespray

About 1yr ago I ran k8s with 300 nodes using kube spray https://github.com/kubernetes-sigs/kubespray . Never had any real issue with it. We did finally move to the cloud though.

I would highly recommend https://github.com/kubernetes-sigs/kubespray