strimzi-kafka-operator VS poc-files

Knative dashboards together with Kafka's dashboards it sheds light on almost any aspect of what's going on in the system.

Strimzi is an operator for deploying Kafka in K8s, and makes extensive use of CRs. E.g., https://github.com/strimzi/strimzi-kafka-operator/blob/0.31.1/examples/kafka/kafka-persistent.yaml

But I encountered a problem. In the example files they provide prometheus-additional.yaml file which is referred to in prometheus.yaml as additionalScrapeConfigs field. It collects Kubernetes node metrics for which it has panels in example Grafana dashboards. All good. But the problem is, the prometheus-additional.yaml file has a field called namespaces: names: [] (File can be seen here - https://github.com/strimzi/strimzi-kafka-operator/blob/main/examples/metrics/prometheus-additional-properties/prometheus-additional.yaml) in kubernetes_sd_configs.

Follow Strimzi quickstart to install kafka in knative-eventing namespace, but use different Kafka cluster definition, see below. Knative workloads are expecting to be run in knative-eventing namespace, otherwise issues arise. And it's easier to keep Knative and Kafka in one namespace. Use kafka-cluster.yaml as kafka cluster resource instead of the one used in Strimzi quickstart (kafka-single-persistent.yaml). If you're not limited on disk, best to set storage: size: 50Gi or 100Gb in kafka-cluster yaml, and at least 25Gb for zookeeper storage. For trial quota, you're limited to 20Gb and 10Gb for zookeeper (if we're doing 2 Kafka clusters, if one - can be more).