prometheus-cpp VS Grafana

Rust does have some very cool mechanisms for safety, including in the presence of concurrency.

But this thread is generating blowback from someone saying: “slow down there with the hand-rolled atomic operations, you can hand-roll your multi-threaded locking strategy and it’ll be way safer at a modest cost!”

So, probably not the target audience for Rust ;)

I use a lot of C++ still because there are libraries I want and I have a significant investment in existing code, but I’d love to get to something more modern.

Hand-rolled atomics and load/store relaxation in application code make even seasoned C++ hackers a bit nervous: we saw this shit from business logic hackers at FB all the time and my colleague coined the term “aggressively intermediate” for the style.

I don’t mean to pick on the author of a quite good library (and it is quite good), but I ran across this the other day:

https://github.com/jupp0r/prometheus-cpp/blob/master/core/sr...

It’s correct (I think, very easy to be wrong about this sort of thing), but what are we measuring here where we can’t delegate that CAS into pthread? Branch mispredictions?

Either threads are fighting over whatever cache line that’s on (exclusive -> invalid -> exclusive -> invalid), or not. If they are, I’ve just deprived the scheduler of the opportunity to wake me up when the other 59 threads are done. If they’re not, I’ve maybe saved like one line in my L1.

And in something like a metrics library, you could be wrong for a very long time before someone pinned it down.

Some metric collecting tool, for example, Prometheus and its client library for C++

You can use https://github.com/jupp0r/prometheus-cpp (your custom data) in combination with https://grafana.com/ that has plugins for things like cpu usage.

Grafana and Prometheus is what I’d use if you’re deploying this in Kubernetes. I haven’t used the C++ library, but generally you can just add a Prometheus client library (e.g. https://github.com/jupp0r/prometheus-cpp ) which makes it quite easy to instrument your application and expose metrics via HTTP. Then you need to configure Prometheus to scrape metrics from your application.

Monitoring application logs is a crucial aspect of the software development and deployment lifecycle. In this post, we'll delve into the process of observing logs generated by Docker container applications operating within HashiCorp Nomad. With the aid of Grafana, Vector, and Loki, we'll explore effective strategies for log analysis and visualization, enhancing visibility and troubleshooting capabilities within your Nomad environment.

To help us visualize these scenarios, we'll build a Grafana Dashboard so we can follow along.

Visualization and Analysis: Choose a tool with intuitive and customizable dashboards, charts, and visualizations. A question to ask is, "Are the visualization features of this tool user-friendly and adaptable to our team's specific needs?" Tools like Grafana and Kibana provide powerful visualization capabilities.

Prometheus: Open-source monitoring system. Often used together with Grafana.

Grafana

Yes I'm aware that Grafana is open source but the method I used to find the API endpoints is far quicker than digging through hundreds of files in a codebase I'm not familiar with.

So, you will add one last container to allow us to visualize this data: Grafana, an open-source analytics and visualization platform that allows us to see traces and metrics simply. You can set Grafana to read data from both Tempo and Prometheus by setting them as datastores with the following grafana.datasource.yaml config file:

In example above, we use 2 additional parameters: code (HTTP response code) and page (page identifier), which provide detailed statistics. For example, you can build such graphs in Grafana:

Robin switched to the Grafana dashboard tab, and sure enough, the 5xx volume on web service was rising. It had not hit the critical alert thresholds yet, but customers had already started noticing.