talos VS k3sup

I think this is underrated as a design flaw for how Linux tends to be used in 2024. At its most benign it's an anachronism and potential source of complexity, as it's worst it's a major source of security flaws and unintended behavior (eg linux multitenancy was designed for two people in the same lab sharing a server, not for running completely untrusted workloads at huge scale).

I haven't had a chance to try it out but this is why I think Talos linux (https://www.talos.dev/) is a step in the right direction for Linux as it is used for cloud/servers. Though personally I think multitenancy esp. regarding containerized applications/cgroups is a bigger problem and I don't know if they're addressing that.

How to create a VM with the Talos image is beyond the scope of this article. Please refer to the official documentation for guidance. After bootstrapping the control plane, the next step is to deploy the Talos CCM along with a CNI plugin.

Maybe in the future I will try others systems, like Talos which is designed for Kubernetes - secure, immutable, and minimal.

I've used Talos Linux[1] on a production infrastructure. To keep a Maintainability. (Because there are no person to maintain a infrastructure 24/7)

All the configurations are made and came from YAML. So I can manage and share on Git. And able to spin a new node (or cluster) ASAP.

For my own, I'm using a NixOS as a daily driver. It's pretty great to spin up machine and environment ASAP. (I don't know why I keep saying `ASAP`, but time is a money.)

However the downside is require a strong knowledge of Nix Language. Sometime the installer crashses.

Without that, it's pretty great.

---

[1]: https://www.talos.dev/

Log aggregation: https://reclaim-the-stack.com/docs/platform-components/log-a...

Observability is on the whole better than what we had at Heroku since we now have direct access to realtime resource consumption of all infrastructure parts. We also have infinite log retention which would have been prohibitively expensive using Heroku logging addons (though we cap retention at 12 months for GDPR reasons).

> Who/What is going to be doing that on this new platform and how much does that cost?

Me and my colleague who created the tool together manage infrastructure / OS upgrades and look into issues etc. So far we've been in production 1.5 years on this platform. On average we spent perhaps 3 days per month doing platform related work (mostly software upgrades). The rest we spend on full stack application development.

The hypothesis for migrating to Kubernetes was that the available database operators would be robust enough to automate all common high availability / backup / disaster recovery issues. This has proven to be true, apart from the Redis operator which has been our only pain point from a software point of view so far. We are currently rolling out a replacement approach using our own Kubernetes templates instead of relying on an operator at all for Redis.

> Now you need to maintain k8s, postgresql, elasticsearch, redis, secret managements, OSs, storage... These are complex systems that require people understanding how they internally work

Thanks to Talos Linux (https://www.talos.dev/), maintaining K8s has been a non issue.

TIL about Talos (https://github.com/siderolabs/talos, via your github/onedr0p/cluster-template link). I'd been previously running k3s cluster on a mixture of x86 and ARM (RPi) nodes, and frankly it was a bit of a PiTA to maintain.

About a month ago I setup a Kubernetes cluster using Talos to handle my container load at home.

I considered deploying Talos a few weeks ago, and I ran into this:

https://github.com/siderolabs/talos/issues/8367

Unless I’ve missed something, this isn’t a big deal in an AWS-style cloud where extra storage volumes (EBS, etc) have essentially no incremental cost, and maybe it’s okay on bare metal if the bare metal is explicitly designed with a completely separate boot disk (this includes Raspberry Pi using SD for boot and some other device for actual storage), but it seemed like a mostly showstopping issue for an average server that was specced with the intent to boot off a partition.

I suppose one could fudge it with NVMe namespaces if the hardware cooperates. (I’ve never personally tried setting up a nontrivial namespace setup.)

I assume https://www.talos.dev/

Basically a small OS that will prop itself up and allow you to create/adopt into a Kubernetes cluster. Seems to work well from my experience and pretty easy to get set up on.

First, obligatory xkcd [0].

> This challenge/solution consumed my entire interest for that day. My dopamine hit was because I wouldn't have to do the BigBoringTask ever again.

Yep. Occasionally I have to stop and remind myself that all I'm trying to do is rename 10 files (for example), and by the time I remember the {ba,z}sh-ism for parameter substitution, I could have probably manually renamed them. I usually tell myself that it's not nearly as fun, though.

This does occasionally present detrimental facets, though. I have a homelab, and as most people with one, its primary purpose is storing and serving media files (I promise I do other things too, but let's be honest – Plex is what people care about). I run apps in K3OS, which has been dead for quite some time. The NAS is in a VM under Proxmox, and I build images with Packer + Ansible. I've been wanting to shift K3OS over to Talos [1] for some time, but I had convinced myself that it was only worthwhile if all of it was in IaC, starting from PXE. I got most of the way there, and then stopped due to work taking more of my life than I wanted. Unfortunately, around this time the NAS broke (as in a hardware failure, not a software issue), and I was refusing to bring it back until the entire homelab was up to my absurd self-imposed standards. Eventually I convinced myself this was a ridiculous punishment, replaced the dead hardware, and brought it back.

[0]: https://xkcd.com/1319/

[1]: https://www.talos.dev/

# Maintainer: Talha Altinel pkgname=k3sup pkgver=0.13.0 pkgrel=1 pkgdesc='A tool to bootstrap K3s over SSH in < 60s' arch=('x86_64') url='https://github.com/alexellis/k3sup' license=('MIT') depends=('glibc' 'openssh') makedepends=('git' 'go>=1.20') source=("${pkgname}-${pkgver}.tar.gz::https://github.com/alexellis/k3sup/archive/${pkgver}.tar.gz") sha256sums=('24939844ac6de581eb05ef6425c89c32b2d0e22800f1344c19b2164eec846c92') _commit=('1d2e443ea56a355cc6bd0a14a8f8a2661a72f2e8') build() { cd "$pkgname-$pkgver" export CGO_CPPFLAGS="${CPPFLAGS}" export CGO_CFLAGS="${CFLAGS}" export CGO_CXXFLAGS="${CXXFLAGS}" export CGO_LDFLAGS="${LDFLAGS}" export GOFLAGS="-buildmode=pie -trimpath -mod=readonly -modcacherw" go build \ -ldflags "-s -w -X github.com/alexellis/k3sup/cmd.Version=$pkgver -X github.com/alexellis/k3sup/cmd.GitCommit=$_commit" \ -o k3sup \ . for shell in bash fish zsh; do ./k3sup completion "$shell" > "$shell-completion" done } package() { cd "$pkgname-$pkgver" install -Dm755 -t "$pkgdir/usr/bin" k3sup mkdir -p "${pkgdir}/usr/share/bash-completion/completions/" mkdir -p "${pkgdir}/usr/share/zsh/site-functions/" mkdir -p "${pkgdir}/usr/share/fish/vendor_completions.d/" install -Dm644 bash-completion "$pkgdir/usr/share/bash-completion/completions/k3sup" install -Dm644 fish-completion "$pkgdir/usr/share/fish/vendor_completions.d/k3sup.fish" install -Dm644 zsh-completion "$pkgdir/usr/share/zsh/site-functions/_k3sup" install -Dm644 -t "$pkgdir/usr/share/licenses/$pkgname" LICENSE }

I recommend learning docker first, then pick a vps host from vpsbenchmarks, then use k3sup to deploy a kubernetes cluster on that, then follow a getting-started kubernetes tutorial from there. You'll also want to buy a domain name with tld-list and then provision a TLS certificate with cert-manager and letsencrypt (skip steps 1-4 because Google Cloud is overpriced).

I just installed k3s yesterday using k3sup on 6 VMs (3 masters, 3 workers) each with 2GB RAM ( limited by the actual RAM on hardware, for now ) with Ubuntu 22.04 as the base OS.

k3s installed with k3sup, longhorn for storage, kube-vip for API VIP, and MetalLB for service load balancer using local subnet, and of course Rancher.

My applause to Alex Ellis for writing a clear, direct call to arms!

Their work is super useful and interesting. I've added them to my list of sponsorships: https://github.com/sponsors/alexellis

Yeah, this is the answer, but I would use this with K3S: https://github.com/alexellis/k3sup

$ curl -sLS https://get.k3sup.dev | sh x86_64 Downloading package https://github.com/alexellis/k3sup/releases/download/0.12.12/k3sup as /home/ec2-user/k3sup Download complete. ============================================================ The script was run as a user who is unable to write to /usr/local/bin. To complete the installation the following commands may need to be run manually. ============================================================ sudo cp k3sup /usr/local/bin/k3sup ================================================================ alexellis's work on k3sup needs your support https://github.com/sponsors/alexellis ================================================================ No nos devolverá nada, pero podremos correr lo siguiente para saber si k3sup efectivamente se instalo: