client_golang VS zap

Prometheus, an open-source systems monitoring and alerting toolkit, has client libraries that allow you to instrument your services in a variety of languages. Here's an example of how you might use the Go client library to expose metrics on an HTTP endpoint. We will leverage the prometheus go client to expose metrics and create them.

Monitoring prometheus, grafana

I'd recommend github.com/victoriametrics/metrics for exporting custom metrics in Prometheus format. It is easier to use than the github.com/prometheus/client_golang.

Start at the official documentation and the Go client library .

DataDog can read Prometheus metrics. Note that this approach works for metrics but not for traces although you can configure DataDog to receive OpenTelemetry traces as well https://github.com/prometheus/client_golang

For Go, the standard is to use Prometheus and the Prometheus Go client library.

The project currently uses slog package from standard library for logging. But switching to a more advanced logger like zap could offer more flexibility and features.

It's nice to have this in the standard library, but it doesn't solve any existing pain points around structured log metadata and contexts. We use zap [0] and store a zap logger on the request context which allows different parts of the request pipeline to log with things like tenantid, traceId, and correlationId automatically appended. But getting a logger off the context is annoying, leads to inconsistent logging practices, and creates a logger dependency throughout most of our Go code.

[0] https://github.com/uber-go/zap

Kubebuilder, like much of the k8s ecosystem, utilizes zap for logging. Out of the box, the Kubebuilder zap configuration outputs a timestamp for each log, which gets formatted using scientific notation. This makes it difficult for me to read the time of an event just by glancing at it. Personally, I prefer ISO 8601, so let's change it!

What else would you expect from a structured logging package?

To me it absolutely makes sense as the default and standard for 99% of applications, and the API isn't much unlike something like Zap[0] (a popular Go structured logger).

The attributes aren't an "arbitrary" concept, they're a completely normal concept for structured loggers. Groups are maybe less standard, but reasonable nevertheless.

I'm not sure if you're aware that this is specifically a structured logging package. There already is a "simple" logging package[1] in the sodlib, and has been for ages, and isn't particularly fast either to my knowledge. If you want really fast you take a library (which would also make sure to optimize allocations heavily).

[0]: https://pkg.go.dev/go.uber.org/zap

[1]: https://pkg.go.dev/log

And finally for structured logging: https://github.com/uber-go/zap

For logging: I recommend using Uber Zap https://github.com/uber-go/zap It will log stack backtraces and makes it super easy to debug errors when deployed. I typically log in the business logic and not below. And log at the entry for failures to start the system. Maybe not necessary for this example, but it’s an essential piece of any API backend.