TimescaleDB VS ClickHouse

> based on the time period

Is it the kind of thing where the TimescaleDB extension would make sense?

https://github.com/timescale/timescaledb

Timescale — Open-source time-series SQL database optimized for fast ingest and complex queries.

Compression and other features use the non-Apache license:

https://github.com/timescale/timescaledb/tree/main/tsl

Don't forget PostgreSQL extensions. For something like a chat log, TimescaleDB (https://www.timescale.com/) can be surprisingly efficient. It will handle partitioning for you, with additional features like data reordering, compression, and retention policies.

Offboard it to Postgres specialists like https://www.timescale.com/

TimescaleDB{:target="_blank"}: an extension of PostgreSQL that adds time-series capabilities to the relational database model. It provides scalability and performance optimizations for handling large volumes of time-stamped data while maintaining the flexibility of a relational database.

Have some experience with https://www.timescale.com in this context

ClickHouse is a high-performance, column-oriented SQL database management system (DBMS) designed for online analytical processing (OLAP). ClickHouse allows using Arrow Stream as an output format.

We started to look for alternatives and quickly landed at Clickhouse.

In today’s story, we will discuss CVE-2024-22412 which affected ClickHouse a popular open-source column-oriented database management system typically used for online analytical processing (OLAP) in real-time. You can find the full write-up of the vulnerability here.

This is really great!

I can suggest more ideas that will be easy to add:

- a spark line or heat map of upvotes for every thread: https://github.com/ClickHouse/ClickHouse/issues/59020

- a built-in SQL editor for custom queries;

If you need help in supporting or hosting it, write to milovidov at clickhouse.com

We chose ClickHouse to store the dictionary as we ran into deadlock and performance issues with Postgres writes as we scaled the number of workers. ClickHouse's async inserts are fantastic for this task and allowed us to ingest the entire 38M+ document dataset in < 1hr.

Minio is fairly easy to setup locally or in CI.

We use it for CI in ClickHouse, for example: https://github.com/ClickHouse/ClickHouse/blob/master/docker/...

PS. If you want to know about this dataset, check https://github.com/ClickHouse/ClickHouse/issues/18842

This is a good article!

In ClickHouse, interval arithmetic is applied to index analysis. A sparse index consists of granules, and each granule is an interval of tuples in lexicographic order. This interval is decomposed into a union of hyperrectangles. Conditions such as comparisons, logic operators, and many other functions are evaluated on these hyperrectangles, yielding boolean intervals. Boolean intervals represent ternary logic (always true, always false, can be true or false). Interesting tricks include: applying functions that are monotonic on ranges (for example, the function "day of month" is monotonic as long as the month does not change), calculating function preimages on intervals, and even calculating preimages of n-ary functions, which is useful for space-filling curves, such as Morton or Hilbert curves.

Check for more details: https://github.com/ClickHouse/ClickHouse/blob/master/src/Sto...

Or see examples, such as https://adsb.exposed/