talos VS rke2

Super cool. I always enjoy reading about systems that challenge, well, "ossified" assumptions. An OS not providing a shell, for example? Madness! ... or is it genius, if the OS has a specific purpose...? It's thought-provoking, if nothing else.

I'm a bit skeptical of parts. For instance, the "init" binary being less than 400 lines of golang - wow! And sure, main.go [1] is less than 400 lines and very readable. Then you squint at the list of imported packages, or look to the left at the directory list and realize main.go isn't nearly the entire init binary.

That `talosctl list` invocation [2] didn't escape my notice either. Sure, the base OS may have only a handful of binaries - how many of those traditional utilities have been stuffed into the API server? Not that I disagree with the approach! I think every company eventually replaces direct shell access with a daemon like this. It's just that "binary footprint" can get a bit funny if you have a really sophisticated API server sitting somewhere.

[1]: https://github.com/siderolabs/talos/blob/main/internal/app/m...

[2]: https://www.talos.dev/v1.6/reference/cli/#talosctl-list

Where `kube.cue` sets reasonable defaults (e.g. image is /). The "cluster" runs on a mini PC in my basement, and I have a small Digital Ocean VM with a static IP acting as an ingress (networking via Tailscale). Backups to cloud storage with restic, alerting/monitoring with Prometheus/Grafana, Caddy/Tailscale for local ingress.

[1] https://www.talos.dev/

[2] https://cuelang.org/

Looks somewhat similar to the talos Linux project[1]

[1] https://www.talos.dev/

Talos Linux basically implements their entire userspace in Go and its similar to BottleRocketOS, because it is designed to host Kubernetes.

https://www.talos.dev/

You might be surprised to find that Talos os (linux distro for kubernetes) mostly uses Go: https://github.com/siderolabs/talos

I've been using a 3 nuc (actually Ryzen devices) k3s on SuSE MicroOS https://microos.opensuse.org/ for my homelab for a while, and I really like it. They made some really nice decisions on which parts of k8s to trim down and which Networking / LB / Ingress to use.

The option to use sqlite in place of etcd on an even lighter single node setup makes it super interesting for even lighter weight homelab container environment setups.

I even use it with Longhorn https://longhorn.io/ for shared block storage on the mini cluster.

If anyone uses it with MicroOS, just make sure you switch to kured https://kured.dev/ for the transactional-updates reboot method.

I'd love to compare it against Talos https://www.talos.dev/ but their lack of support for a persistent storage partition (only separate storage device) really hurts most small home / office usage I'd want to try.

If you’re interested in something not AWS check out Talos https://www.talos.dev/

It’s been around longer than Bottlerocket

Can't talk about work, but my homelab is Azure and Oracle managed k8s (AKS/OKE), with onprem Talos soon (Turing Pi 2). My Flux monorepo has the details. OKE performs noticably worse (update cycle, features, control plane performance), but it provides 4 ARM cores and 24GB RAM free so I can't complain

Check out Talos Linux , it essentially solves a lot of the OS level challenges because there is no hardening requirements, the whole OS is API driven and you can bootstrap a cluster using their CLI. The entire node config is basically just a YAML file so can be managed via a GitOps workflow, then you can just layer something like Flux or Argo on top.

To add to the other excellent answers, I would recommend adding Bottlerocket to your reading list: https://github.com/bottlerocket-os/bottlerocket#readme

I'm also aware of (but haven't used) https://github.com/siderolabs/talos#readme

I just realized your question may have implied a desktop os, whereas Bottlerocket, Flatcar, and likely the others in this specific thread are server-side. I don't have much experience with trying to solve that problem on the desktop except for the horror-show that is snap

Go to "https://github.com/rancher/rke2/releases" and select a release. After that, you should download rke2-images-core.linux-amd64.tar.gz. Create the Rancher data directory to put the compressed offline images.

In this essay, we showed how to use Rancher rke2 to deploy a Kubernetes cluster with 6 Debian nodes with firewall enabled. We've also covered deploying Cilium as a CNI for our cluster and have it completely replace kube-proxy so as to increase speed and gain more observability via Cilium tools. This article also showed how to deploy Metallb to manage IP pools and load balance traffic for those IP pools. Throughout this guide, we assumed that we have an external load balancer that will distribute traffic to our workload and control plane nodes. For further information please visit rke2 official documents: "https://docs.rke2.io/".

In this blog post, we will demonstrate how easy and fast it is to deploy Sveltos on an RKE2 cluster with the help of ArgoCD, register two RKE2 Cluster API (CAPI) clusters and create a ClusterProfile to deploy Prometheus and Grafana Helm charts down the managed CAPI clusters.

Did something happen to the Apache 2 rancher? https://github.com/rancher/rancher/blob/v2.7.5/LICENSE RKE2 is similarly Apache 2: https://github.com/rancher/rke2/blob/v1.26.7%2Brke2r1/LICENS...

To provision all of my clusters, I use Rancher with RKE2. The primary Rancher server is hosted on a bootstrapped RKE2 cluster running on a VPS.

Golang has burned me more than once with bizarre design decisions that break things in a user hostile way.

The last one we ran into was a change in Go 1.15 where servers that presented a TLS certificate with the hostname encoded into the CN field instead of the more appropriate SAN field always fail validation.

The behavior could be disabled however that functionality was removed in 1.18 with no way to opt back into the old behavior. I understand why SAN is the right way to do it but in this case I didn’t control the server.

Developers at Google probably never have to deal with 3rd parties with shitty infrastructure but a lot of us do.

Here’s a bug in rke that’s related https://github.com/rancher/rke2/issues/775

Currently at https://docs.rke2.io/ it says (excerpted):

just looking at this myself. I think k3s has more support for arm, but looking through the github repo there are a lot of bugs indicating its a mess. RKE2 seems to be their big push, they also have a github issue open that has been open for the last 2 releases that they are going to add a update path from k3s to rke2. https://github.com/rancher/rke2/issues/881

RKE (https://rancher.com/docs/rke) and RKE2 (https://docs.rke2.io/) from Rancher folks

RKE2 - Rancher's Next Generation Kubernetes Distribution