vcluster VS hierarchical-namespaces

Kubernetes? You mean the container orchestration system where they forgot to add Multi-tenancy? And no namespaces are not Multi-tenancy...

https://www.vcluster.com/

Whats the difference between this and vcluster (https://github.com/loft-sh/vcluster)?

Yep, as we see it they compliment each other quite well. DevPod takes your workspace to the cloud and DevSpace let's you develop against your Kubernetes cluster - potentially the same one you used to start your workspace.

Internally we use both in our development setup, spinning up remote workspaces using DevPod, installing DevSpace and kind into the devcontainer, then using DevSpace to develop against the cluster. See the vcluster setup[1] as an example

[1]https://github.com/loft-sh/vcluster/tree/main/.devcontainer

The tenants are internal dev teams so yeah maybe not. I was considering multi-tenanting different environments isolated at the kube layer with vCluster and have the vCluster pods running in Kata containers giving maximum isolation but still having a single management cluster. Ideally also avoiding the need to buy a second set of hardware for a dev environment

Vcluster

Looks interesting. How does this solution compare to vcluster?

sounds like the best option for you , is a tool called VCluster by loft ( https://www.vcluster.com/) , this way you can install as many k8s cluster as you want in the same k8s host cluster , those cluster share workers nodes and networking, but each has a separated "api server" , so it looks like you have a dedicated cluster with their own namespaces and tools . take a look at the docs to get a better understanding and how they work.

We're about to run a PoC with vcluster for isolated sandboxes, this might be relevant to you too

I think namespaces, RBAC and network policies are sufficient to partition users from the same organisation. I would investigate the use of vcluster ig you want to give your users even more isolation and capability (such as installing CRDs)

Depends on the use case. Namespaces provides soft isolation (so it means they share same Apiserver, PV's and global resources such as CRD's), but can be restricted with network policies. So it means, there's still potential in breaking other namespaces if you change PV's or CRD's which are used by other namespaces. Multi-Cluster solution can provide full isolation, but its also really expensive in resource consumption and maintenance/management effort. If namespaced-isolation isnt enough for your use case, you can consider vclusters (https://www.vcluster.com/)

HNC_VERSION=v1.1.0 HNC_VARIANT=default kubectl apply -f https://github.com/kubernetes-sigs/hierarchical-namespaces/releases/latest/download/hnc-manager.yaml

Kubernetes has a lot of limitations from a multi tenancy perspective.

It's functional, but I think it's not as polished as the rest of Kubernetes which is why Kubernetes has a multi tenancy SIG that spawned the hierarchical namespace controller (https://github.com/kubernetes-sigs/hierarchical-namespaces) and virtual clusters (https://github.com/kubernetes-sigs/cluster-api-provider-nest...)

Kyverno's a great option. Depending on the usecase you might want to consider https://github.com/kubernetes-sigs/hierarchical-namespaces as well (disclaimer: I'm the original author) - it's good if groups of related namespaces need related objects.

❯ HNC_VERSION=v1.0.0 ❯ kubectl apply -f https://github.com/kubernetes-sigs/hierarchical-namespaces/releases/download/${HNC_VERSION}/default.yaml namespace/hnc-system created customresourcedefinition.apiextensions.k8s.io/hierarchyconfigurations.hnc.x-k8s.io created customresourcedefinition.apiextensions.k8s.io/hncconfigurations.hnc.x-k8s.io created customresourcedefinition.apiextensions.k8s.io/subnamespaceanchors.hnc.x-k8s.io created role.rbac.authorization.k8s.io/hnc-leader-election-role created clusterrole.rbac.authorization.k8s.io/hnc-admin-role created clusterrole.rbac.authorization.k8s.io/hnc-manager-role created clusterrole.rbac.authorization.k8s.io/hnc-proxy-role created rolebinding.rbac.authorization.k8s.io/hnc-leader-election-rolebinding created clusterrolebinding.rbac.authorization.k8s.io/hnc-manager-rolebinding created clusterrolebinding.rbac.authorization.k8s.io/hnc-proxy-rolebinding created secret/hnc-webhook-server-cert created service/hnc-controller-manager-metrics-service created service/hnc-webhook-service created deployment.apps/hnc-controller-manager created mutatingwebhookconfiguration.admissionregistration.k8s.io/hnc-mutating-webhook-configuration created validatingwebhookconfiguration.admissionregistration.k8s.io/hnc-validating-webhook-configuration created # Install helper plugin ❯ kubectl krew install hns

I would say it’s an anti-pattern since using a namespace for multiple environments will be a pain. Not sure what you mean by CRDs though. There is an addon that gives you namespace hierarchies. I.e. each team gets a namespace and they can have sub-namespaces for environments. Check it out: https://github.com/kubernetes-sigs/hierarchical-namespaces

Looking into the Hierarchical Namespace Controller to see if it can simplify our heavily multi-tenanted clusters. So far so good!

HNC is designed for these kinds of scenarios: https://github.com/kubernetes-sigs/hierarchical-namespaces

Project HNC