promscale VS signoz

Now that Promscale has been deprecated, what are the other ideal means of self-hosted long term Prometheus storage?

Oh wow, I browsed the project just a few weeks ago, didn't see it then. I see the deprecation is recent (https://github.com/timescale/promscale/issues/1836)

They say:

> if you want to have a seamless experience between metrics and traces, then current experience of stitching together Prometheus & Jaeger is not great.

But I wonder if using Promscale https://github.com/timescale/promscale would make Prometheus & Jaeger not such a big problem as SigNoz imply.

Promscale readme:

> Promscale is a unified metric and trace observability backend for Prometheus, Jaeger and OpenTelemetry built on PostgreSQL and TimescaleDB.

Either way, SigNoz seems interesting indeed. And am glad to see that SigNoz supports OpenTelemetry.

Hi! So the team is over 100 at this point, but engineering effort is spread across multiple products at this point.

The core timescaledb repo [0] has 10-15 primary engineers (although we are aggressively hiring for database internal engineers), with a few others working on DB hyperfunctions and our function pipelining [1] in a separate extension [2]. I think generally the set of folks who contribute to low-level database internals in C is just smaller than other type of projects.

We also have our promscale product [3], which is our observability backend powered by SQL & TimescaleDB.

And then there is Timescale Cloud, which is obviously a large engineering effort (most of which does not happen in public repos).

And we are hiring. Fully remote & global.

https://www.timescale.com/careers

[0] https://github.com/timescale/timescaledb

[1] https://www.timescale.com/blog/function-pipelines-building-f...

[2] https://github.com/timescale/timescaledb-toolkit

[3] https://github.com/timescale/promscale ; https://github.com/timescale/tobs

Promscale is a connector for Prometheus, one of the leading open-source monitoring solutions. Promscale is developed by Timescale, a time series database with full compatibility to Postgres. Since logs are time series events, Timescale developed Promscale to ingest events from Prometheus and make them available in SQL. You can install Promscale in numerous ways.

In case it helps, Promscale (from Timescale) offers long-term storage for Prometheus data and supports both PromQL and SQL queries. Here's the project page: https://www.timescale.com/promscale/ and the repo is here https://github.com/timescale/promscale It also support OpenTelemetry tracing if that's of interest.

At first, let's give the definition of `time series`. This is a series of (timestamp, value) pairs ordered by timestamp. The `value` may contain arbitrary data - a floating-point value, a text, a json, a data structure with many columns, etc. Each time series is uniquely identified by its name plus an optional set of {label="value"} labels. For example, temperature{city="London",country="UK"} or log_stream{host="foobar",datacenter="abc",app="nginx"}.

ClickHouse is perfectly optimized for storing and querying of such time series, including metrics. That's true that ClickHouse isn't optimized for handling millions of tiny inserts per second. It prefers infrequent batches with big number of rows per each batch. But this isn't the real problem in practice, because:

1) ClickHouse provides Buffer table engine for frequent inserts.

2) It is easy to create a special proxy app or library for data buffering before sending it to ClickHouse.

TimescaleDB provides Promscale [1] - a service, which allows using TimescaleDB as a storage backend for Prometheus. Unfortunately, it doesn't show outstanding performance comparing to Prometheus itself and to other remote storage solutions for Prometheus. Promscale requires more disk space, disk IO, CPU and RAM according to production tests [2], [3].

[1] https://github.com/timescale/promscale

[2] https://abiosgaming.com/press/high-cardinality-aggregations/

[3] https://valyala.medium.com/promscale-vs-victoriametrics-reso...

Full disclosure: I'm CTO at VictoriaMetrics - competing solution for TimescaleDB. VictoriaMetrics is built on top of architecture ideas from ClickHouse.

Promscale + TimescaleDB

Jaeger is great for local debugging—but when you move to production, you need more: metrics, logs, alerting, dashboards. That’s where SigNoz comes in.

check out SigNoz: https://github.com/SigNoz/signoz

We started building signoz as an OS alternative of Datadog/New Relic four years back and opentelemetry-native from day 1. We have shipped some good features on top of Opentelemetry and because of OTel's semantic conventions & our query builder, you can correlate any telemetry across signals.

sudo usermod -aG docker $USER newgrp docker docker -v git clone -b main https://github.com/SigNoz/signoz.git && cd signoz/deploy/ ./install.sh docker ps

Hey folks! I’m a maintainer at SigNoz[0] , an open-source observability platform

Looking to get some feedback on my observations on querying for o11y and if this resonates with more folks here

I feel that current observability tooling significantly lags behind user expectations by failing to support a critical capability: querying across different telemetry signals.

This limitation turns what should be powerful correlation capabilities into mere “correlation theater”, a superficial simulation of insights rather than true analytical power.

Here’s the current gaps I see

1/ Suppose I want to retrieve logs from the host which have the highest CPU in the last 13 minutes. It’s not possible to query this seamlessly today unless you query the metrics first and paste the results into logs query builder and retrieve your results. Seamless correlation across signal querying is nearly impossible today.

2/ COUNT distinct on multiple columns is not possible today. Most platforms let you perform a count distinct on one col, say count unique of source OR count unique of host OR count unique of service etc. Adding multiple dimensions and drilling down deeper into this is also a serious pain-point.

and some points on how we at SigNoz are thinking these gaps can be addressed,

1/ Sub-query support: The ability to use the results of one query as input to another, mainly for getting filtered output

2/ Cross-signal joins: Support for joining data across different telemetry signals, for seeing signals side-by-side along with a couple of more stuff.

Early thoughts in this blog[1], what do you think? does it resonate or seems like a use case not many ppl have?

[0] https://github.com/signoz/signoz

git clone -b main https://github.com/SigNoz/signoz.git cd signoz/deploy/ ./install.sh

Check out SigNoz, https://github.com/signoz/signoz

Has metrics, logs and traces in a single app and built natively on OpenTelemetry

Disclaimer : I am a maintainer