kube-state-metrics VS kube-bench

git clone https://github.com/kubernetes/kube-state-metrics.git

https://github.com/kubernetes/kube-state-metrics/blob/main/docs/pod-metrics.md According to this github repo completion is responsible of termination date if I correctly understood .

kube-state-metrics

Those are entirely different types of data. You can get that from something like kube-state-metrics

I've never had a cluster of that size, so take it with a grain of salt - but maybe you could try purpose-based sharding? KSM has allowlist and denylist config flags, for configuring which metrics it exposes https://github.com/kubernetes/kube-state-metrics/blob/main/docs/cli-arguments.md

Kubernetes itself will not notify you, the way I've seen people do this, is to use something like kube-state-metrics or node_exporter, export that to Prometheus (or preferrably VictoriaMetrics because Prometheus is terrible IMO), and then setup alarms on that with alertmanager or equivalent, or just look at dashboards regularly with Grafana. Realistically I recommend only setting alerts on disk usage and application/database latency. CPU and memory utilization isn't a great metric to alert on a lot of the time.

Another tip that you could consider spelling out a little more, is to monitor the number of resources created by Kind. This is somewhat mentioned for jobs and services, but any Kind of which thousands of resources are created will put stress on the control-plane. The total number of resources per namespace/cluster can be monitored with kube-state-metrics. KSM does not emit metrics of resources created from CRDs. These metrics can be implemented with KSM's custom resource state metrics: https://github.com/kubernetes/kube-state-metrics/blob/main/docs/customresourcestate-metrics.md

We then have various other Metrics called Kubernetes Workload Metrics. These are the dashboards with names that start with “Kubernetes / Compute Resources / Workload”. These dashboards are specific to the services you are running. They take into account the Kubernetes Workloads in your various namespaces, using kube-state-metrics. For a closer look, check out otel_demo_app_k8s_dashboard.tf.

The first step to optimizing costs is gaining visibility into your costs using tools. Kubernetes provides a Metrics Server and kube-state-metrics that can give you the overall picture of resource utilization by your cluster. There are more tools that provide more granular breakdowns and provide dashboards with business metrics, infra cost, and alerting functionalities. Here are some strategies to optimize your resource utilization and cloud bills on k8s.

However, no matter how well our applications are secured, the security of our entire IT environment ultimately depends on the security of our infrastructure. Therefore, in the lab to follow, we will shift our focus away from Kubernetes workloads and instead explore how we can evaluate and improve upon the security of our Kubernetes clusters with kube-bench, the industry-leading Kubernetes benchmarking solution developed by Aqua.

The official repository can be found here with detailed installation instructions.

curl -L [https://github.com/aquasecurity/kube-bench/releases/download/v0.6.10/kube-bench_0.6.10_linux_amd64.deb](https://github.com/aquasecurity/kube-bench/releases/download/v0.6.2/kube-bench_0.6.2_linux_amd64.deb) -o kube-bench_0.6.10_linux_amd64.deb

I haven't used Kubernetes for a while, but shouldn't kube-bench (1) be enough? Do you have to check anything manually?

(1) https://github.com/aquasecurity/kube-bench

kube-bench checks whether Kubernetes is deployed securely by running the checks documented in the CIS Kubernetes Benchmark. We can deploy kube-bench as a Job that runs daily and consume its report in CI/CD to pass or fail the pipeline based on the level of severity.

kube-bench can be executed as a simple command on the host, as a container on the host using Docker command, or as a job inside Kubernetes Cluster. In case it is run inside a container/pod, it will need access to the PID namespace of the host system. The methods to run kube-bench in AKS, EKS, GKE, On-prem cluster, Openshift and ACK (Alibaba Cloud Container Service For Kubernetes) are different but well documented.

Check this out: kube-bench - CIS Benchmark

Kube-bench, written as a Go application, is deployable as a container. Ready-made job.yaml files make it easy to run Kube-bench inside a Kubernetes cluster or on a managed Kubernetes service, such as Azure Kubernetes Service (AKS), Amazon Elastic Kubernetes Service (EKS), Google Kubernetes Engine (GKE), or OpenShift. Here's a link to Kube-Bench on Github

kube-bench - Checks whether Kubernetes is deployed securely by running CIS Kubernetes Benchmark