tobs VS mtail

Hi! So the team is over 100 at this point, but engineering effort is spread across multiple products at this point.

The core timescaledb repo [0] has 10-15 primary engineers (although we are aggressively hiring for database internal engineers), with a few others working on DB hyperfunctions and our function pipelining [1] in a separate extension [2]. I think generally the set of folks who contribute to low-level database internals in C is just smaller than other type of projects.

We also have our promscale product [3], which is our observability backend powered by SQL & TimescaleDB.

And then there is Timescale Cloud, which is obviously a large engineering effort (most of which does not happen in public repos).

And we are hiring. Fully remote & global.

https://www.timescale.com/careers

[0] https://github.com/timescale/timescaledb

[1] https://www.timescale.com/blog/function-pipelines-building-f...

[2] https://github.com/timescale/timescaledb-toolkit

[3] https://github.com/timescale/promscale ; https://github.com/timescale/tobs

You can do that with something like mtail. Basically write expressions that match your logs and produce metrics.

mtail is a standard tool for this.

I use goflow2 to do something like this. I don't specifically use syslog itself for this, but mtail to generate the metrics.

I can recommend mtail. Here is a good example nginx script.

If not, https://github.com/google/mtail might be a good option.

Count the lines with mtail. You can regexp match the values out into labels.

If you want to extract whitebox metrics from logs, maybe all you need is mtail.

You can use mtail (https://github.com/google/mtail) for this. You'll need to figure out how to plug it into your setup, but mtail will do the metrics from logs thing.

Convert your logs to metrics with mtail.

For nginx, you'll need to setup a log parser like mtail because it doesn't really have much for metrics to begin with.