kubespray VS ingress-nginx

When it came to deployment, I had several options, and I chose the hard way: deploying Kubernetes on bare metal nodes using KubeSpray. Troubleshooting bare metal Kubernetes deployments honed my skills in pinpointing issues. This hands-on experience provided a deep understanding of how each component, like the Control Plane, kubelet, Container Runtime, and scheduler, interacts to orchestrate containers.

Worth noting that if you like ansible, Kubespray has had documented air-gap installation since 2018 https://github.com/kubernetes-sigs/kubespray/commit/963c3479...

calico v3.7.3 kubernetes v1.16.0 installed via kubespray https://github.com/kubernetes-sigs/kubespray

About 1yr ago I ran k8s with 300 nodes using kube spray https://github.com/kubernetes-sigs/kubespray . Never had any real issue with it. We did finally move to the cloud though.

I would highly recommend https://github.com/kubernetes-sigs/kubespray

We build our own distro called Compliant Kubernetes, and we use (a fork of) kubespray to install the base Kubernetes layer. Our distro is entirely open source, so you can use it as a reference, if you want.

If you want to run Kubernetes on VM's you can look into kubespray and play around with the options for installing and checking the configuration after the fact.

# Add NGINX ingress using Helm eks.HelmChart( self, "NginxIngress", cluster=cluster, chart="ingress-nginx", repository="https://kubernetes.github.io/ingress-nginx", namespace="ingress-nginx", values=helm_values )

ingress-nginx

The second one is a combination of tools: External DNS, cert-manager, and NGINX ingress. Using these as a stack, you can quickly deploy an application, making it available through a DNS with a TLS without much effort via simple annotations. When I first discovered External DNS, I was amazed at its quality.

resource "helm_release" "icrelease" { name = "nginx-ingress" repository = "https://kubernetes.github.io/ingress-nginx" chart = "ingress-nginx" version = "4.9.1" namespace = kubernetes_namespace.icnamespace.metadata[0].name set { name = "controller.ingressClassResource.default" value = "true" } }

helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx helm repo update helm upgrade --install ingress-nginx ingress-nginx/ingress-nginx --namespace ingress-nginx --create-namespace -f custom/ghost/nginx.yaml

For example, if you chose Nginx Ingress, you will use some of its dozens of annotations that are not portable if you decide to switch to another Ingress implementation like Apache APISIX.

The controller is installed with helm upgrade --install ingress-nginx ingress-nginx --repo https://kubernetes.github.io/ingress-nginx --namespace ingress-nginx --create-namespace

helm repo add jetstack https://charts.jetstack.io helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx helm repo add rancher-latest https://releases.rancher.com/server-charts/latest helm repo update helm repo list

Everything in the YAML snippets below — except for ingress configuration — relates to configuring the NGINX ingress controller. This includes customizing the default configuration.