timescale-analytics VS TimescaleDB

Hi! So the team is over 100 at this point, but engineering effort is spread across multiple products at this point.

The core timescaledb repo [0] has 10-15 primary engineers (although we are aggressively hiring for database internal engineers), with a few others working on DB hyperfunctions and our function pipelining [1] in a separate extension [2]. I think generally the set of folks who contribute to low-level database internals in C is just smaller than other type of projects.

We also have our promscale product [3], which is our observability backend powered by SQL & TimescaleDB.

And then there is Timescale Cloud, which is obviously a large engineering effort (most of which does not happen in public repos).

And we are hiring. Fully remote & global.

https://www.timescale.com/careers

[0] https://github.com/timescale/timescaledb

[1] https://www.timescale.com/blog/function-pipelines-building-f...

[2] https://github.com/timescale/timescaledb-toolkit

[3] https://github.com/timescale/promscale ; https://github.com/timescale/tobs

(NB: Post author here)

This is in the TimescaleDB Toolkit extension [1] which is licensed under our community license for now and it's not available on DO. It is available on our cloud service fully managed. You can also install it and run it for free yourself.

[1]: https://github.com/timescale/timescaledb-toolkit

Absolutely! We're actually developing a lot of that: https://github.com/timescale/timescaledb-toolkit/tree/main/d...

A number of the things you're looking for we've done experimentally and we'll be stabilizing over the next few releases. So we'd love some feedback while we're still able to futz with the API without making breaking changes.

But the two you're asking about are, I think, going to be covered by hyperloglog (we just reimplemented the internals with HLL++) and stats_agg family of functions, which have both 1D (which will give you avg, stddev, variance, etc) and 2D (co-variance, slope, intercept, x-intercept etc as well as all the 1D functions).

Would also love issues if you think we're missing other stuff, going to be generalizing this and want to make it useful for folks.

(NB: Post author here.)

If you know that you're going to be doing downsampling at the hourly level then a continuous aggregate on the hour is probably a good idea. We're also building some functions to make some of the continuous aggregate stuff for these sorts of cases easier/more accurate in more cases, especially if you need things like exact averages when you don't have the same number of points in an hour and want to re-aggregate on top of the continuous agg. See: https://github.com/timescale/timescale-analytics/pull/141/files

Fair point about adaptive chunking. You sound like a long-term user!

There is always a trade-off between getting features to users quickly to experiment and incrementally improve, versus doing it always very conservatively.

When we launched adaptive chunking (introduced in 0.11, deprecated in 1.2), we explicitly marked it as beta and default off, to hopefully reflect that. [1]

The approach we are now taking with Timescale Analytics [2] is to have an explicit distinction between experimental features (which will be part of a distinct"experimental" schema in the database, and must be expressly turned on with appropriate warnings) and stable features. Hopefully this can help find a good balance between stability and velocity, but feedback welcome!

[1] https://github.com/timescale/timescaledb/releases/tag/0.11.0

[2] https://github.com/timescale/timescale-analytics/tree/main/e...

Don't forget PostgreSQL extensions. For something like a chat log, TimescaleDB (https://www.timescale.com/) can be surprisingly efficient. It will handle partitioning for you, with additional features like data reordering, compression, and retention policies.

Offboard it to Postgres specialists like https://www.timescale.com/

TimescaleDB{:target="_blank"}: an extension of PostgreSQL that adds time-series capabilities to the relational database model. It provides scalability and performance optimizations for handling large volumes of time-stamped data while maintaining the flexibility of a relational database.

Have some experience with https://www.timescale.com in this context

What about getting in touch with commercial organisations that have products/services based on PostgreSQL? For example Timescale, EDB, and Citus Data, or really any hosting provider that offers a managed PostgreSQL service.

There is Timescale.

Friends don't let their friends choose Mysql :)

A super long time ago (decades) when I was using Oracle regularly I had to make a decision on which way to go. Although Mysql then had the mindshare I thought that Postgres was more similar to Oracle, more standards compliant, and more of a real enterprise type of DB. The rumor was also that Postgres was heavier than MySQL. Too many horror stories of lost data (MyIsam), bad transactions (MyIsam lacks transaction integrity), and the number of Mysql gotchas being a really long list influenced me.

In time I actually found out that I had underestimated one of the most important attributes of Postgres that was a huge strength over Mysql: the power of community. Because Postgres has a really superb community that can be found on Libera Chat and elsewhere, and they are very willing to help out, I think Postgres has a huge advantage over Mysql. RhodiumToad [Andrew Gierth] https://github.com/RhodiumToad & davidfetter [David Fetter] https://www.linkedin.com/in/davidfetter are incredibly helpful folks.

I don't know that Postgres' licensing made a huge difference or not but my perception is that there are a ton of 3rd party products based on Postgres but customized to specific DB needs because of the more liberalness of the PG license which is MIT/BSD derived https://www.postgresql.org/about/licence/

Some of the PG based 3rd party DBs:

Enterprise DB https://www.enterprisedb.com/ - general purpose PG with some variants

Greenplum https://greenplum.org/ - Data warehousing

Crunchydata https://www.crunchydata.com/products/hardened-postgres - high security Postgres for regulated environments

Citus https://www.citusdata.com - Distributed DB & Columnar

Timescale https://www.timescale.com/

Why Choose PG today?

If you want better ACID: Postgres

If you want more compliant SQL: Postgres

If you want more customizability to a variety of use-cases: Postgres using a variant

If you want the flexibility of using NOSQL at times: Postgres

If you want more product knowledge reusability for other backend products: Postgres

TimescaleDB is Postgres with extensions to automatically partition tables for fast processing of time series data.