longhorn VS nfs-subdir-external-provisioner

For my primary CSI provider, I use Longhorn, and to administrate my clusters, I use the Kubernetes Dashboard. In front of all of my private dashboards, I use Traefik Forward Auth to limit who can access them.

Maybe with https://longhorn.io

For data must be protected, I use Longhorn https://longhorn.io/ which is basically iSCSI/NFS with replication.

Backups are definitely necessary. The problem though is that you cant detect by default if you are backing up corrupted data. By now i found three approaches: 1. Use a filesystem that detects and fixes errors (e. g. btrfs or zfs) 2. Use par2 (parchive) to generate checksums on your files manually 3. I found the following issue on the k8s storage solution I currently use: https://github.com/longhorn/longhorn/issues/3198 For now I am waiting for option 3 getting merged. In the long run I will upgrade my laptop to btrfs and for now use par2. Hope that provides a starting point. Feel free to add your ideas!

You can use the Longhorn iSCSI installer script as an example. https://github.com/longhorn/longhorn/blob/master/deploy/prerequisite/longhorn-iscsi-installation.yaml

Rancher's Longhorn (even though it's only a single-node)

Application storage now: yes, NAS is probably the best way to go about it. I believe k3s uses some kind of "local storage" by default but there's plenty of alternative Container Storage Interfaces (CSI) out there. Longhorn worked well for me and it covers most uses cases I care about: snapshots, backups, and replication. I don't know about any "NAS providers" though, and once again there will be performance drawbacks with having your storage that externalized.

For me longhorn works really well as my storage backend.

Longhorn is a lightweight, reliable, and powerful distributed block storage system for Kubernetes.

You mentioned storage... That's always tricky but provisioning persistent volumes with https://longhorn.io is really simple... It's also easy to setup, the helm chart is wonderful

Depends on how much resiliency you need . Something like https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner works well for a lab or non-prod cluster. You could even use something like this in prod if you have access to highly reliably NFS mounts.

I used the NFS provisioner for persistent volumes until I got the Ceph side up and running. I created a share on my NAS specifically for k8s. It worked very well and had the bonus of being just a regular file system that you could browse/edit easily (just place files in or edit config). I would agree with not moving plex into k8s. I right now just have a barebones 1 control 2 worker setup using k3s.

This is what you need https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner Point it to a folder and it will create subfolders for each PVC.

nfs-subdir-external-provisioner: https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner

With NFS I was able to get that to work using NFS SubDir Provisioner and this is working fine as far as I can tell. Though it is odd how it works. You dont seem to be able to tell the exact path you're going for, so you have to create a provisioner that goes to the path directly and even it will create its own folder with a GUI associated to it. This really isnt idea for things like scripted backups, or even knowing how persistence works properly.

I have also used: https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner for playing with NFS, directly from K8, instead of hosts.

Then nfs is the way to go. Im using this one https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner at home with k3s. works fine for my usecases. I havent tested snapshots and restores yet.

so I'm in Cybersecurity right now.. but I'm trying to pivot my career and I'm just in love with K8s / I think its a good future proof tech to learn. my current lab is 3x physical nodes running vmware (VMUG) then I built some ubuntu vms and installed microk8s. Everything seems to work really well. I had/have nfs-subdir-external-provision up and running. I've also tested it so that if I drain a node the pod is recreated on another node just fine and the pv/c works great. I've been trying to figure out why this is a bad solution vs using longhorn/OpenEBS. Sounds like NFS vs iSCSI is the answer.