opentelemetry-python-contrib VS Grafana

In my last blog post, I showed y’all how to instrument Python code with OpenTelemetry (OTel), à la auto-instrumentation. You may also recall from that post that I recommended using the Python auto-instrumentation binary even for non-auto-instrumented libraries, because it abstracts all that pesky OTel config stuff so nicely. When you use it, along with any applicable Python auto-instrumentation libraries (installed courtesy of opentelemetry-bootstrap), it takes care of context propagation across related services for you.

More specifically, auto-instrumentation uses shims or bytecode instrumentation agents to intercept your code at runtime or at compile-time to add tracing and metrics instrumentation to the libraries and frameworks you depend on. The beauty of auto-instrumentation is that it requires a minimum amount of effort. Sit back, relax, and enjoy the show. A number of popular Python libraries are auto-instrumented, including Flask and Django. You can find the full list here.

And sure, there's gaps and those are awful when you get to them. But writing minimal tracing integration is pretty easy. This is the full source of the psycopg2 instrumentation. https://github.com/open-telemetry/opentelemetry-python-contrib/blob/main/instrumentation/opentelemetry-instrumentation-psycopg2/src/opentelemetry/instrumentation/psycopg2/__init__.py

Monitoring application logs is a crucial aspect of the software development and deployment lifecycle. In this post, we'll delve into the process of observing logs generated by Docker container applications operating within HashiCorp Nomad. With the aid of Grafana, Vector, and Loki, we'll explore effective strategies for log analysis and visualization, enhancing visibility and troubleshooting capabilities within your Nomad environment.

To help us visualize these scenarios, we'll build a Grafana Dashboard so we can follow along.

Visualization and Analysis: Choose a tool with intuitive and customizable dashboards, charts, and visualizations. A question to ask is, "Are the visualization features of this tool user-friendly and adaptable to our team's specific needs?" Tools like Grafana and Kibana provide powerful visualization capabilities.

Prometheus: Open-source monitoring system. Often used together with Grafana.

Grafana

Yes I'm aware that Grafana is open source but the method I used to find the API endpoints is far quicker than digging through hundreds of files in a codebase I'm not familiar with.

So, you will add one last container to allow us to visualize this data: Grafana, an open-source analytics and visualization platform that allows us to see traces and metrics simply. You can set Grafana to read data from both Tempo and Prometheus by setting them as datastores with the following grafana.datasource.yaml config file:

In example above, we use 2 additional parameters: code (HTTP response code) and page (page identifier), which provide detailed statistics. For example, you can build such graphs in Grafana: