kube-prometheus VS ingress-nginx

The last one is mostly an observability stack with Prometheus, Metric server, and Prometheus adapter to have excellent insights into what is happening on the cluster. You can reuse the same stack for autoscaling by repurposing all the data collected for monitoring.

kustomize Kube Prometheus

On the other hand, the Kube-prometheus project provides documentation and scripts to operate end-to-end Kubernetes cluster monitoring using the Prometheus Operator, making easier the process of monitoring the Kubernetes cluster.

For our load testing we’ve deployed Temporal on Kubernetes, and we’re using MySQL for the persistence backend. The MySQL instance has 4 CPU cores and 32GB RAM, and each Temporal service (Frontend, History, Matching, and Worker) has 2 pods, with requests for 1 CPU core and 1GB RAM as a starting point. We’re not setting CPU limits for our pods—see our upcoming Temporal on Kubernetes post for more details on why. For monitoring we’ll use Prometheus and Grafana, installed via the kube-prometheus stack, giving us some useful Kubernetes metrics.

The 2 most common approaches I have seen are kube-prometheus-stack and kube-prometheus..

- I'm missing a lot of the Grafana dashboards that are provisioned during the deployment, not sure why as it has worked before, and wanted to add them after install... I believe it's different ConfigMaps like the one in kube-prometheus but I was wondering if there's a way to force provisioning them all again at once (multiple k8s, node_exporter, vm, etc)?

Check out the kube Prometheus project -- https://github.com/prometheus-operator/kube-prometheus It's a bit heavy, but the included recording rules and dashboards give you a great start at understanding your cluster.

The actual link to the prometheus/grafana bundle: https://github.com/prometheus-operator/kube-prometheus

Here’s a list of what’s installed: https://github.com/prometheus-operator/kube-prometheus/tree/main/manifests

# Add NGINX ingress using Helm eks.HelmChart( self, "NginxIngress", cluster=cluster, chart="ingress-nginx", repository="https://kubernetes.github.io/ingress-nginx", namespace="ingress-nginx", values=helm_values )

ingress-nginx

The second one is a combination of tools: External DNS, cert-manager, and NGINX ingress. Using these as a stack, you can quickly deploy an application, making it available through a DNS with a TLS without much effort via simple annotations. When I first discovered External DNS, I was amazed at its quality.

resource "helm_release" "icrelease" { name = "nginx-ingress" repository = "https://kubernetes.github.io/ingress-nginx" chart = "ingress-nginx" version = "4.9.1" namespace = kubernetes_namespace.icnamespace.metadata[0].name set { name = "controller.ingressClassResource.default" value = "true" } }

helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx helm repo update helm upgrade --install ingress-nginx ingress-nginx/ingress-nginx --namespace ingress-nginx --create-namespace -f custom/ghost/nginx.yaml

For example, if you chose Nginx Ingress, you will use some of its dozens of annotations that are not portable if you decide to switch to another Ingress implementation like Apache APISIX.

The controller is installed with helm upgrade --install ingress-nginx ingress-nginx --repo https://kubernetes.github.io/ingress-nginx --namespace ingress-nginx --create-namespace

helm repo add jetstack https://charts.jetstack.io helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx helm repo add rancher-latest https://releases.rancher.com/server-charts/latest helm repo update helm repo list

Everything in the YAML snippets below — except for ingress configuration — relates to configuring the NGINX ingress controller. This includes customizing the default configuration.