mtail VS goflow2

You can do that with something like mtail. Basically write expressions that match your logs and produce metrics.

mtail is a standard tool for this.

I use goflow2 to do something like this. I don't specifically use syslog itself for this, but mtail to generate the metrics.

I can recommend mtail. Here is a good example nginx script.

If not, https://github.com/google/mtail might be a good option.

Count the lines with mtail. You can regexp match the values out into labels.

If you want to extract whitebox metrics from logs, maybe all you need is mtail.

You can use mtail (https://github.com/google/mtail) for this. You'll need to figure out how to plug it into your setup, but mtail will do the metrics from logs thing.

Convert your logs to metrics with mtail.

For nginx, you'll need to setup a log parser like mtail because it doesn't really have much for metrics to begin with.

I use goflow2 to do something like this. I don't specifically use syslog itself for this, but mtail to generate the metrics.

I've been using this one: https://github.com/netsampler/goflow2

Take a look at goflow2 which is based on Cloudflare's flow analysis pipeline.

I'm not particularly familiar with netflow formats, but this collector emits Prometheus metrics.

GoFlow2 is a high-performance sFlow/IPFIX/NetFlow collector that gathers network information and serializes it in a common format. This fork of CloudFlare's GoFlow provides horizontal scalability, consistent format and the ability to work with raw samples and build aggregation and custom enrichment. Our thanks for this suggestion goes to SuperQue.

You want something like goflow2.

There's goflow2, which is a fork of CloudFlare's goflow netflow collector.

I'd suggest looking at goflow2 before you write something new.

I personally use goflow2, but my needs are pretty simple.