longhorn VS nfs-subdir-external-provisioner

You can use node selector rules to force Longhorn to use some nodes for storage. You can also add taints to create dedicated storage nodes. Please see the GitHub link listed below. https://github.com/longhorn/longhorn/issues/1633

In Longhorn GUI the volume shows as Detached. It would attach if I do is manually but that makes no impact to the Pod. I've seen people having similar issues in 2614 and have tried with create-default-disk-on-labeled-nodes but that does not seem to work the way I would like to. If I deploy without nodeSelector and disable scheduling on worker nodes it works fine.

rook/longhorn for distributed storage

Longhorn is exactly what you're looking for. It's easy to set up and backs up to NFS.

More than performance, I am worried about security. They don't seem to have considered it at all when building it: https://github.com/longhorn/longhorn/issues/1983

Probably not a good recommendation until this hole is plugged.

They've also got Longhorn, a distributed container-attached storage solution that's very simple to understand and easy to deploy. Performance is another thing but that's the same with all of the general networked storage solutions (Ceph included).

Rancher's got a well deserved good impression in my mind, though early on I avoided it since it seemed like they were building a walled garden.

[0]: https://github.com/longhorn/longhorn

All of their projects have been fully open, monetized with enterprise support. They straight up fully donated their storage project to the CNCF. They've been good stewards of all the tools so far with ones like k3s greatly benefitting the community. And have put a decent amount of effort into putting resources supporting issues on even non-paid issues on the GitHub projects.

Also just in case the above wasn't enough I'm probably actually going to run Longhorn because it's just SO much easier than Ceph to manage and gives me the availability for a ~40% perf loss AFAIK.

Longhorn for Kubernetes storage. It lets me automatically provision PersistentVolumes and snapshot/back them up to S3. I've read that other things such as Ceph and Rook are more stable and I've definitely been bit by some non-critical bugs, but the project is very active and bugs are squashed quickly (though they don't release very frequently). This is the gem I can't live without, being able to snapshot, backup and restore things like my Plex volume has saved me at least a dozen times already.

Another consideration is using something like https://longhorn.io/, it's a bit like ceph, and supports mirroring over two nodes

I have also used: https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner for playing with NFS, directly from K8, instead of hosts.

Then nfs is the way to go. Im using this one https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner at home with k3s. works fine for my usecases. I havent tested snapshots and restores yet.

Does the NFS mount exist as a StorageClass in your cluster? If not you'll need to set it up first. Kubernetes doesn't provide an internal NFS provisioner so you'll need to use an external one, something like this. We don't support doing this directly via Portainer as of yet, but you should be able to set the provisioner up using a manifest (as it is an application, after all).

NFS will work just fine. You can create PersistentVolumes manually to mount paths on the NFS to Pods or if you want dynamic provisioning check this.

so I'm in Cybersecurity right now.. but I'm trying to pivot my career and I'm just in love with K8s / I think its a good future proof tech to learn. my current lab is 3x physical nodes running vmware (VMUG) then I built some ubuntu vms and installed microk8s. Everything seems to work really well. I had/have nfs-subdir-external-provision up and running. I've also tested it so that if I drain a node the pod is recreated on another node just fine and the pv/c works great. I've been trying to figure out why this is a bad solution vs using longhorn/OpenEBS. Sounds like NFS vs iSCSI is the answer.

Forgive me if I'm comparing the wrong tools, I'm just starting my CKA journey and I'm at the point of learning about Persistent volumes / claims. I can set them up just fine but digging around I've found that if you're in a multi node cluster you will run into an issue if Pod A is built on Node A and then Node A goes down.. the PV/C will be unreachable from POD A on Node B. So I found what seems like a common solution: kubernetes-sigs/nfs-subdir-external-provisioner . I have this up and running and it works great. I deployed a NGINX pod on Node A.. then drained Node A and it rebuilt on Node B working just fine using the same PV/C. My question is.. is LongHorn.io a better solution? More commonly used? Are these two completely different things?

I use an nfs server with both standard nfs pv and pvc for shared storage between deployments and https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner for all others.

https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner with an external NFS server.

Persistent Storage: https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner