helm-charts VS kustomize

The go to solution for a k8s monitoring setup is https://github.com/prometheus-community/helm-charts/tree/mai...

That's why Kube-Prometheus-Stack was created. It installs a collection of Kubernetes manifests, Grafana dashboards, and Prometheus rules, providing an easy-to-operate, end-to-end Kubernetes cluster monitoring solution with Prometheus using the Prometheus Operator.

Step 1: Installing Kube-prometheus-stack

Kubernetes Documentation: https://kubernetes.io/docs/home/ Kubernetes Tutorials: https://kubernetes.io/docs/tutorials/ Kubernetes Community: https://kubernetes.io/community/ Prometheus: https://prometheus.io/ Grafana: https://grafana.com/ Elasticsearch: https://www.elastic.co/elasticsearch/ Kibana: https://www.elastic.co/kibana Helm: https://helm.sh/ Prometheus Helm Chart: https://github.com/prometheus-community/helm-charts/tree/main/prometheus Grafana Helm Chart: https://github.com/grafana/helm-charts/tree/main/grafana Elasticsearch Helm Chart: https://github.com/elastic/helm-charts/tree/main/elasticsearch Kibana Helm Chart: https://github.com/elastic/helm-charts/tree/main/kibana RBAC: https://kubernetes.io/docs/reference/access-authn-authz/rbac/ Network Policies: https://kubernetes.io/docs/concepts/services-networking/network-policies/ StatefulSets: https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/ DaemonSets: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/ Taints and Tolerations: https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/ Custom Resource Definitions (CRDs): https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/ Operators: https://kubernetes.io/docs/concepts/extend-kubernetes/operator/

Installation example for prometheus:

Prometheus can be deployed using the Prometheus Helm Chart. This helm chart contains a lot of features such as the already mentioned Push Gateway, Alert Manager and so on. For simplicity reasons of this tutorial I will not show all the Helm chart configuration but you can see a real example used by me here.

Building upon Bartłomiej Płotka's insightful blog on Prometheus and its passthrough agent mode, this post dives into implementing multi-cluster Prometheus support. Notably, the official inclusion of support in the widely-used kube-prometheus-stack came with the release in July 2023, making it easier to extend Prometheus monitoring across clusters.

The first thing you'll need, of course, is at least one backend to store metrics. So install Prometheus now:

When we update any configuration files (index.html), the configMap created by configMapGenerator will have a new hash, and applying new configuration will ask the deployment to use the new configmap, but the old configmap (with the old hash string) will not removed. You can read more about this here.

kustomize: brew install kustomize

It's a built in feature of kustomize https://github.com/kubernetes-sigs/kustomize/blob/master/api/types/helmchartargs.go

I think the combination of Kustomize and helm works in my experience. For advanced use cases, you can also see KRM functions in Kustomize.

See for instance a related issue: https://github.com/kubernetes-sigs/kustomize/issues/3866

Using a strategic merge is the safest way so you avoid the index fragility.

Rendering out the manifests is something I have been pushing for. Not having to understand how every templating tool works and what actually is being changed is key. Though, it gets complicated when you use helm (or any templating/patching tool) that produces many variants. You also lose any release/deployment time hooks that are provided (helm hooks or recently "patched" kustomize env variables).

https://github.com/kubernetes-sigs/kustomize/blob/master/examples/chart.md (edit: oh I see the other commenter also included this link, oops)