zap_otlp VS zap

Zap OTLP is an OpenTelemetry Protocol (OTLP) encoder and sync for Zap - a popular structured logging library for Go.

Here's the Github repo - https://github.com/SigNoz/zap_otlp/

You can use this plugin to send Zap logs to any observability backend which can understand OTLP format.

Zap OTLP is designed to enhance logging capabilities in Go applications by enabling the transmission of logs from the Zap logger to an OpenTelemetry Protocol (OTLP) endpoint. This integration allows developers to seamlessly incorporate their application logs into any tool that supports OTLP.

The plugin provides a straightforward setup process where users can create two encoders: one for console output and another for OTLP.

It also includes functionality to initialize an OTLP sync, which handles the actual sending of log data to the specified OTLP endpoint. The sync can be configured with options such as batch size, resource schema, and service name, allowing for customization of how logs are grouped and identified.

Additionally, the plugin offers features like batching logs for efficient transmission, setting batch intervals, and the ability to include trace context in log entries.

This makes it particularly useful for applications that require detailed logging with distributed tracing capabilities.

The plugin also supports secure connections and can be easily configured to work with any OpenTelemetry native observability backend like SigNoz.

Do have a look and let us know what you think? Feel free to start discussions in the repo

1. You're right! Will fix it to handle this as well as the support for relative directories

2. Yes, Will integrate a logging library instead of fmt (https://github.com/uber-go/zap)

The project currently uses slog package from standard library for logging. But switching to a more advanced logger like zap could offer more flexibility and features.

It's nice to have this in the standard library, but it doesn't solve any existing pain points around structured log metadata and contexts. We use zap [0] and store a zap logger on the request context which allows different parts of the request pipeline to log with things like tenantid, traceId, and correlationId automatically appended. But getting a logger off the context is annoying, leads to inconsistent logging practices, and creates a logger dependency throughout most of our Go code.

[0] https://github.com/uber-go/zap

Kubebuilder, like much of the k8s ecosystem, utilizes zap for logging. Out of the box, the Kubebuilder zap configuration outputs a timestamp for each log, which gets formatted using scientific notation. This makes it difficult for me to read the time of an event just by glancing at it. Personally, I prefer ISO 8601, so let's change it!

What else would you expect from a structured logging package?

To me it absolutely makes sense as the default and standard for 99% of applications, and the API isn't much unlike something like Zap[0] (a popular Go structured logger).

The attributes aren't an "arbitrary" concept, they're a completely normal concept for structured loggers. Groups are maybe less standard, but reasonable nevertheless.

I'm not sure if you're aware that this is specifically a structured logging package. There already is a "simple" logging package[1] in the sodlib, and has been for ages, and isn't particularly fast either to my knowledge. If you want really fast you take a library (which would also make sure to optimize allocations heavily).

[0]: https://pkg.go.dev/go.uber.org/zap

[1]: https://pkg.go.dev/log

And finally for structured logging: https://github.com/uber-go/zap

For logging: I recommend using Uber Zap https://github.com/uber-go/zap It will log stack backtraces and makes it super easy to debug errors when deployed. I typically log in the business logic and not below. And log at the entry for failures to start the system. Maybe not necessary for this example, but it’s an essential piece of any API backend.