operator VS runbooks

There is also victoriametrics-operator and victoriametrics k8s stack helm chart.

Try VictoriaMetrics operator for k8s, which would store data to VictoriaMetrics DBaaS at AWS.

For metrics there is VictoriaMetrics operator and helm chart, which can be used for monitoring Kubernetes. See this guide.

will fit both bare metals and k8s environment;

There is also https://github.com/VictoriaMetrics/operator

Just use Grafana Loki for logs and victoriametrics operator for metrics.

was designed to run on premise, but also can run in k8s via k8s operator.

There are also good patterns for monitoring the status of jobs.

Check out this job monitoring pattern.

For a freely available resource, I found GitLab’s Runvbooks a good example to strive towards. But this is the output of a large team and several years. My advice is to just start with something simple.

It's a weighted combination of all services. The SLA weightings for all service areas are defined in the gitlab.com service catalog.

Take a look at this pushgateway-based job monitoring pattern.

I want to use the GitLab dashboards here in my local Grafana environment - https://gitlab.com/gitlab-com/runbooks/-/tree/master/dashboards

Also, take a look at the code https://gitlab.com/gitlab-com/runbooks

GitLab puts everything in a git repo in markdown. A single repo that contains a few different things. All of the production alerts are sourced from there. This makes it possible to review alerts and runbook changes in the same code review.