longhorn VS k3sup

Distributed storage systems enable us to store data that can be made available clusterwide. Excellent! But dynamically apportioning storage across a multi-node cluster is a very complex job. So this is another area where Kubernetes typically outsources the job to plugins (e.g. Cloud providers like Azure or AWS, or systems like Rook or Longhorn).

Since I want to play with Kubernetes anyway, I'll set up a k8s cluster. It will have 2 master and 4 worker nodes. Each VM will have 4 cores, 8 GB of RAM, a 32 GB root virtual disk, and a 250 GB data virtual disk for Longhorn volumes. I'll create an ansible user via cloud-init and allow access via SSH.

That's a sweet setup.

Have you come across Longhorn[0]?

I wanted to have a look at that for storage when I was using Pis as it theoretically should be lighter-weight than Ceph, who knows. Didn't get around to it though.

[0] https://longhorn.io/

Dan: Argo CD is the first tool I install. For AWS, I will add Karpenter to manage costs. I will also use Longhorn for on-prem storage solutions, though I'd need ingress. Depending on the situation, I will install Argo CD first and then one of those other two.

I overcome this issue with Longhorn which is a native distributed block storage for Kubernetes and supports by default RWM and not only RWO (ReadWriteOnce).

I'm looking forward to Longhorn[1] taking advantage of this technology.

[1]: https://github.com/longhorn/longhorn

I've been using a 3 nuc (actually Ryzen devices) k3s on SuSE MicroOS https://microos.opensuse.org/ for my homelab for a while, and I really like it. They made some really nice decisions on which parts of k8s to trim down and which Networking / LB / Ingress to use.

The option to use sqlite in place of etcd on an even lighter single node setup makes it super interesting for even lighter weight homelab container environment setups.

I even use it with Longhorn https://longhorn.io/ for shared block storage on the mini cluster.

If anyone uses it with MicroOS, just make sure you switch to kured https://kured.dev/ for the transactional-updates reboot method.

I'd love to compare it against Talos https://www.talos.dev/ but their lack of support for a persistent storage partition (only separate storage device) really hurts most small home / office usage I'd want to try.

Hi,i was wondering the same. Found more information here in this document: https://github.com/longhorn/longhorn/blob/v1.5.x/enhancements/20230103-recurring-snapshot-cleanup.md

# Maintainer: Talha Altinel pkgname=k3sup pkgver=0.13.0 pkgrel=1 pkgdesc='A tool to bootstrap K3s over SSH in < 60s' arch=('x86_64') url='https://github.com/alexellis/k3sup' license=('MIT') depends=('glibc' 'openssh') makedepends=('git' 'go>=1.20') source=("${pkgname}-${pkgver}.tar.gz::https://github.com/alexellis/k3sup/archive/${pkgver}.tar.gz") sha256sums=('24939844ac6de581eb05ef6425c89c32b2d0e22800f1344c19b2164eec846c92') _commit=('1d2e443ea56a355cc6bd0a14a8f8a2661a72f2e8') build() { cd "$pkgname-$pkgver" export CGO_CPPFLAGS="${CPPFLAGS}" export CGO_CFLAGS="${CFLAGS}" export CGO_CXXFLAGS="${CXXFLAGS}" export CGO_LDFLAGS="${LDFLAGS}" export GOFLAGS="-buildmode=pie -trimpath -mod=readonly -modcacherw" go build \ -ldflags "-s -w -X github.com/alexellis/k3sup/cmd.Version=$pkgver -X github.com/alexellis/k3sup/cmd.GitCommit=$_commit" \ -o k3sup \ . for shell in bash fish zsh; do ./k3sup completion "$shell" > "$shell-completion" done } package() { cd "$pkgname-$pkgver" install -Dm755 -t "$pkgdir/usr/bin" k3sup mkdir -p "${pkgdir}/usr/share/bash-completion/completions/" mkdir -p "${pkgdir}/usr/share/zsh/site-functions/" mkdir -p "${pkgdir}/usr/share/fish/vendor_completions.d/" install -Dm644 bash-completion "$pkgdir/usr/share/bash-completion/completions/k3sup" install -Dm644 fish-completion "$pkgdir/usr/share/fish/vendor_completions.d/k3sup.fish" install -Dm644 zsh-completion "$pkgdir/usr/share/zsh/site-functions/_k3sup" install -Dm644 -t "$pkgdir/usr/share/licenses/$pkgname" LICENSE }

I recommend learning docker first, then pick a vps host from vpsbenchmarks, then use k3sup to deploy a kubernetes cluster on that, then follow a getting-started kubernetes tutorial from there. You'll also want to buy a domain name with tld-list and then provision a TLS certificate with cert-manager and letsencrypt (skip steps 1-4 because Google Cloud is overpriced).

I just installed k3s yesterday using k3sup on 6 VMs (3 masters, 3 workers) each with 2GB RAM ( limited by the actual RAM on hardware, for now ) with Ubuntu 22.04 as the base OS.

k3s installed with k3sup, longhorn for storage, kube-vip for API VIP, and MetalLB for service load balancer using local subnet, and of course Rancher.

My applause to Alex Ellis for writing a clear, direct call to arms!

Their work is super useful and interesting. I've added them to my list of sponsorships: https://github.com/sponsors/alexellis

Yeah, this is the answer, but I would use this with K3S: https://github.com/alexellis/k3sup

$ curl -sLS https://get.k3sup.dev | sh x86_64 Downloading package https://github.com/alexellis/k3sup/releases/download/0.12.12/k3sup as /home/ec2-user/k3sup Download complete. ============================================================ The script was run as a user who is unable to write to /usr/local/bin. To complete the installation the following commands may need to be run manually. ============================================================ sudo cp k3sup /usr/local/bin/k3sup ================================================================ alexellis's work on k3sup needs your support https://github.com/sponsors/alexellis ================================================================ No nos devolverá nada, pero podremos correr lo siguiente para saber si k3sup efectivamente se instalo: