longhorn VS k3sup

You can use node selector rules to force Longhorn to use some nodes for storage. You can also add taints to create dedicated storage nodes. Please see the GitHub link listed below. https://github.com/longhorn/longhorn/issues/1633

In Longhorn GUI the volume shows as Detached. It would attach if I do is manually but that makes no impact to the Pod. I've seen people having similar issues in 2614 and have tried with create-default-disk-on-labeled-nodes but that does not seem to work the way I would like to. If I deploy without nodeSelector and disable scheduling on worker nodes it works fine.

rook/longhorn for distributed storage

Longhorn is exactly what you're looking for. It's easy to set up and backs up to NFS.

More than performance, I am worried about security. They don't seem to have considered it at all when building it: https://github.com/longhorn/longhorn/issues/1983

Probably not a good recommendation until this hole is plugged.

They've also got Longhorn, a distributed container-attached storage solution that's very simple to understand and easy to deploy. Performance is another thing but that's the same with all of the general networked storage solutions (Ceph included).

Rancher's got a well deserved good impression in my mind, though early on I avoided it since it seemed like they were building a walled garden.

[0]: https://github.com/longhorn/longhorn

All of their projects have been fully open, monetized with enterprise support. They straight up fully donated their storage project to the CNCF. They've been good stewards of all the tools so far with ones like k3s greatly benefitting the community. And have put a decent amount of effort into putting resources supporting issues on even non-paid issues on the GitHub projects.

Also just in case the above wasn't enough I'm probably actually going to run Longhorn because it's just SO much easier than Ceph to manage and gives me the availability for a ~40% perf loss AFAIK.

Longhorn for Kubernetes storage. It lets me automatically provision PersistentVolumes and snapshot/back them up to S3. I've read that other things such as Ceph and Rook are more stable and I've definitely been bit by some non-critical bugs, but the project is very active and bugs are squashed quickly (though they don't release very frequently). This is the gem I can't live without, being able to snapshot, backup and restore things like my Plex volume has saved me at least a dozen times already.

Another consideration is using something like https://longhorn.io/, it's a bit like ceph, and supports mirroring over two nodes

Also, k3sup (https://github.com/alexellis/k3sup) makes it even easier!

Learning Kubernetes (K8s) for me was about getting pissed off enough at Bezos for all the money he was making off of me lol. I wanted to have my own little infrastructure that I control but without having to babysit it. I tried Docker Swarm, which was nice but wasn’t translating into professional-grade cluster experience. I found a project called Open FAAS that let me deploy my own Lambda/serverless. It ran on K8s. So, I followed a few tutorials on K3s (a lightweight version of K8s for the pi) and deployed it with K3sup. The OpenFAAS project, K8s, and Docker are written in Golang, so I learned some of that with A Tour Of Go. After branching out into the K8s world for a bit, I decided to get my Certified Kubernetes Application Developer (CKAD). I took this Udemy course and passed the test on the first go. You’ll learn just about everything you’ll need to know about K8s from that experience.

GitHub - alexellis/k3sup: bootstrap Kubernetes with k3s over SSH < 1 min 🚀

Deploy k3s with k3sup. You'll be up and running in no time!

For other configurations such as adding external database or load balancers, please refer to this github repository.

k3s is far easier to deal w/ for home. Use k3sup to set it up - https://github.com/alexellis/k3sup. Running 6 node RPis w/ ArgoCD, cert-manager, helm etc. Not as hard as people make it out but just like anything demands you respect it and grok it.

You can bootstrap k3s quickly using k3sup !

Thanks for mentioning K3sup [0]

I used Microk8s on a client project late last year and it was really painful, but I am sure it serves a particular set of users who are very much into the Snap/Canonical ecosystem.

In contrast, K3s is very light-weight and can be run in a container via the K3d project.

If folks want to work with K8s upstream or development patches against Kubernetes, they will probably find that KinD is much quicker and easier.

Minikube has also got a lot of love recently, and can run without having a dependency on Virtual Box too.

[0] https://k3sup.dev/