HTML Olap Projects

ClickBench

68 570 9.0 HTML

ClickBench: a Benchmark For Analytical Databases

Project mention: Loading a trillion rows of weather data into TimescaleDB | news.ycombinator.com | 2024-04-16

TimescaleDB primarily serves operational use cases: Developers building products on top of live data, where you are regularly streaming in fresh data, and you often know what many queries look like a priori, because those are powering your live APIs, dashboards, and product experience.
That's different from a data warehouse or many traditional "OLAP" use cases, where you might dump a big dataset statically, and then people will occasionally do ad-hoc queries against it. This is the big weather dataset file sitting on your desktop that you occasionally query while on holidays.
So it's less about "can you store weather data", but what does that use case look like? How are the queries shaped? Are you saving a single dataset for ad-hoc queries across the entire dataset, or continuously streaming in new data, and aging out or de-prioritizing old data?
In most of the products we serve, customers are often interested in recent data in a very granular format ("shallow and wide"), or longer historical queries along a well defined axis ("deep and narrow").
For example, this is where the benefits of TimescaleDB's segmented columnar compression emerges. It optimizes for those queries which are very common in your application, e.g., an IoT application that groups by or selected by deviceID, crypto/fintech analysis based on the ticker symbol, product analytics based on tenantID, etc.
If you look at Clickbench, what most of the queries say are: Scan ALL the data in your database, and GROUP BY one of the 100 columns in the web analytics logs.
- https://github.com/ClickHouse/ClickBench/blob/main/clickhous...
There are almost no time-predicates in the benchmark that Clickhouse created, but perhaps that is not surprising given it was designed for ad-hoc weblog analytics at Yandex.
So yes, Timescale serves many products today that use weather data, but has made different choices than Clickhouse (or things like DuckDB, pg_analytics, etc) to serve those more operational use cases.

github-explorer

13 129 4.3 HTML

Everything You Always Wanted To Know About GitHub (But Were Afraid To Ask)

Project mention: Backdoor in upstream xz/liblzma leading to SSH server compromise | news.ycombinator.com | 2024-03-29

clickhouse has pretty good github_events dataset on their playground that folks can use to do some research - some info on the dataset https://ghe.clickhouse.tech/
Example of what user JiaT75 did so far:
https://play.clickhouse.com/play?user=play#U0VMRUNUICogRlJPT...
pull requests mentioning xz, 5.6 without downgrade, cve being mentioned in the last 60 days:
https://play.clickhouse.com/play?user=play#U0VMRUNUIGNyZWF0Z...

InfluxDB

www.influxdata.com sponsored

Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).