opentelemetry-dotnet VS Grafana

open-telemetry/opentelemetry-dotnet: The OpenTelemetry .NET Client (github.com)

💡Good to know -- If you wish to export traces to Jaeger, you should use the AddJaegerExporter instead of the AddOtlpExporter. Visit the opentelemetry-dotnet repository to see how it's done.

We're going to use OpenTelemetry .NET SDK. Add following nuget dependencies to the project:

We instrument the application with the OpenTelemetry SDK and SqlClient instrumentation library for .NET. First, we add the following NuGet package references to the API’s project file:

It should be handled automatically by OpenTelemetry middlewares. Just look through documentation and samples https://github.com/open-telemetry/opentelemetry-dotnet

Besides that, the propagation fields (traceparent and tracestate) were added in the message header. In the last article, I said that the standard (in the Working Draft (WD) step of the w3c process) recommends to add the propagation fields in the application-properties section by the message publisher. For the current example, I chose to propagate that context in the message header even for AMQP calls as was done in the dotnet OpenTelemetry example. It's important to reinforce that Trace Context: AMQP protocol is not a W3C Recommendation yet. Take a look at the place where the propagation fields were added:

As long as the Azure Functions app knows the instrumentation key from an Application Insights instance, it traces almost everything. OpenTelemetry.NET is one of the Open Telemetry implementations, has recently released v1.0 for tracing. Both metrics and logging are close to GA. However, it doesn't work well with Azure Functions. Therefore, in this post, let's manually implement the tracing at the log level, which is sent to Application Insights.

Monitoring application logs is a crucial aspect of the software development and deployment lifecycle. In this post, we'll delve into the process of observing logs generated by Docker container applications operating within HashiCorp Nomad. With the aid of Grafana, Vector, and Loki, we'll explore effective strategies for log analysis and visualization, enhancing visibility and troubleshooting capabilities within your Nomad environment.

To help us visualize these scenarios, we'll build a Grafana Dashboard so we can follow along.

Visualization and Analysis: Choose a tool with intuitive and customizable dashboards, charts, and visualizations. A question to ask is, "Are the visualization features of this tool user-friendly and adaptable to our team's specific needs?" Tools like Grafana and Kibana provide powerful visualization capabilities.

Prometheus: Open-source monitoring system. Often used together with Grafana.

Grafana

Yes I'm aware that Grafana is open source but the method I used to find the API endpoints is far quicker than digging through hundreds of files in a codebase I'm not familiar with.

So, you will add one last container to allow us to visualize this data: Grafana, an open-source analytics and visualization platform that allows us to see traces and metrics simply. You can set Grafana to read data from both Tempo and Prometheus by setting them as datastores with the following grafana.datasource.yaml config file:

In example above, we use 2 additional parameters: code (HTTP response code) and page (page identifier), which provide detailed statistics. For example, you can build such graphs in Grafana:

Robin switched to the Grafana dashboard tab, and sure enough, the 5xx volume on web service was rising. It had not hit the critical alert thresholds yet, but customers had already started noticing.