pushgateway VS prometheus

Pushgateway

In case the Custom Metric is related to a job or a short-living script, consider using Pushgateway.

So my idea is to have a python script that buckets data and then publishes it again using pushtogateway: https://github.com/prometheus/pushgateway

Is the cron job with a script for getting metric and pushing via https://github.com/prometheus/pushgateway the right way for this task?

The push gateway is a relatively simple service. There’s one main entry point file (main.go) and an api file (api.go) both at 279 lines log. There’s probably a dozen or so api endpoints, but the key one is the one where you post metrics, defined here

We are currently evaluating timeseries databases to build an internal tool for DORA metrics and just looking at prometheus. As tracking releases is more of a push based use case, I looked at pushgateway and the docs stated the following "The Pushgateway is not an event store. [...] tracking something like release events has to happen with some event-logging framework." here

Checked this out? https://github.com/prometheus/pushgateway

While I somehow understand Prometheus idea that pull is easier to scale than push I've had a bad luck with it.

First of all Prometheus doesn't even consider monitoring of long-running jobs other than pull way (which didn't make sense for me). There is push gateway [0] but clients libraries seem to consider it only for short-lived jobs where you can send the metrics at the end [1]. It seems I couldn't "push" from long living jobs trivially

Second when using it for example with django you have to be careful with how you handle multiprocessing that UWSGI/gunicorn does, see [2] - it has bitten me at leas once.

Comparing to push model where I can just push metrics to [3] statsd_exporter directly and be done with it, but support for statsd is lacking both in terms of frameworks (everyone seems to be migrating to native clients...) and functionality (you've to do labeling basically manually [4])

To sum up: Prometheus is really great when it works, until you try to go off-track (intentionally or not, see django [2]) then you see its all undiscovered and immature landscape

[0] https://github.com/prometheus/pushgateway

WriteRequest::timeseries is a vector (https://github.com/prometheus/prometheus/blob/main/prompb/re...) and

Developers widely use Prometheus as a system for operational monitoring and alerting for their projects. Here is a list of tools for monitoring frontend services with Prometheus.

Just to give an example of the power of Go for CLI builds, you may have already used or at least heard of Docker, Kubernetes, Prometheus, Terraform, but what do they all have in common? They all have a large part of their usability via CLI and are developed in Go 🐿.

Distributed system administrators need mechanisms and tools for monitoring individual nodes in order to analyze the system and promptly detect anomalies. Developers also need effective mechanisms for analyzing, diagnosing issues, and identifying bugs in protocol implementations. Logging, tracing, and collecting metrics are common observability techniques to allow monitoring and obtaining diagnostic information from the system; most of the explored code bases use these techniques. OpenTelemetry and Prometheus are popular open-source monitoring solutions, which are used in many of the explored code bases.

Setting up monitoring for a system, especially one involving GRPC communication, provides crucial visibility into its operations. In this guide, we walked through the steps to instrument both a GRPC server and client with Prometheus metrics, exposed those metrics via an HTTP endpoint, and visualized them using Grafana. The Docker-Compose setup simplified the deployment of both Prometheus and Grafana, ensuring a streamlined process.

Alerting and Notification: Select a tool with flexible alerting mechanisms to proactively detect anomalies or deviations from defined thresholds. Consider asking questions like "Does this tool offer customizable alerting options and support notification channels that suit our team's communication preferences?" A tool like Prometheus provides robust alerting capabilities.

Prometheus

Prometheus is an open-source monitoring and alerting toolkit. It is designed to monitor highly dynamic containerized systems, making it an excellent choice for monitoring Docker containers and Kubernetes clusters.

wget https://github.com/prometheus/prometheus/releases/download/v2.46.0/prometheus-2.46.0.linux-amd64.tar.gz

Prometheus: Open-source monitoring system. Often used together with Grafana.