starrocks
ClickBench
Our great sponsors
starrocks | ClickBench | |
---|---|---|
12 | 68 | |
7,764 | 570 | |
4.9% | 8.2% | |
10.0 | 9.0 | |
4 days ago | 9 days ago | |
Java | HTML | |
Apache License 2.0 | GNU General Public License v3.0 or later |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
starrocks
-
A MySQL compatible database engine written in pure Go
tidb has been around for a while, it is distributed, written in Go and Rust, and MySQL compatible. https://github.com/pingcap/tidb
Somewhat relatedly, StarRocks is also MySQL compatible, written in Java and C++, but it's tackling OLAP use-cases. https://github.com/StarRocks/starrocks
- StarRocks – sub-second MPP OLAP database for full analytics scenarios
-
Let's Talk about Joins
I think you're talking about doing denormalization before importing data into an OLAP system to avoid subsequent joins. However, this greatly limits the flexibility of data modeling. Moreover, denormalization can be a headache-inducing process. In fact, I have tested StarRocks (https://github.com/StarRocks/starrocks), and it is capable of performing joins while streaming data imports, and the speed is very fast. It's worth giving it a try.
-
Ask HN: Are there any notable Chinese FLOSS projects?
https://github.com/apache/doris Is a great example. Same for it's cousin https://github.com/StarRocks/starrocks that was an early fork of the doris project.
To be fair, these are the only examples I can think of and I only learned of these as I'm standing up new data infra using starrocks.
-
Open Source Columnar Databases
ClickHouseClickHouse and Starrocks are similar. They are both columnar databases powered by vectorization tech, which means they are really fast.
-
Ask HN: Do you use any software (mainly) developed in China?
StarRocks, it’s a Linux Foundation project now, but a lot of the initial team and community behind it came from China.
https://github.com/StarRocks/starrocks
Funny that I hadn’t heard of them in the database space till they showed up at the top of ClickBench. Makes me wonder what other interesting projects I’m missing out on in China.
- Anyone using StarRocks DB instead of ClickHouse?
-
Show HN: A benchmark for analytical databases (Snowflake, Druid, Redshift)
Full disclosure - I work for StarRocks (starrocks.com)
First of all, this is great. Transparent and healthy competition is always great for the customers!
Regarding the joined table queries that are missing in the tests, this is exactly why we built StarRocks - to give people the best performance of complex analytics queries on both joined tables and single tables.
I encourage you to checkout this blog: https://starrocks.medium.com/starrocks-outperforms-clickhous...
And, give us a star if you think we are doing the right thing: https://github.com/StarRocks/starrocks
Follow us on LinkedIn for the latest updates: https://www.linkedin.com/company/starrocks
- We are looking for a very fast database for big data analysis, does anyone know about starrocks, I heard it is very fast
- wow, i found a super fast database for Big Data analytics,it's called StarRocks,come and take a look!
ClickBench
-
Loading a trillion rows of weather data into TimescaleDB
TimescaleDB primarily serves operational use cases: Developers building products on top of live data, where you are regularly streaming in fresh data, and you often know what many queries look like a priori, because those are powering your live APIs, dashboards, and product experience.
That's different from a data warehouse or many traditional "OLAP" use cases, where you might dump a big dataset statically, and then people will occasionally do ad-hoc queries against it. This is the big weather dataset file sitting on your desktop that you occasionally query while on holidays.
So it's less about "can you store weather data", but what does that use case look like? How are the queries shaped? Are you saving a single dataset for ad-hoc queries across the entire dataset, or continuously streaming in new data, and aging out or de-prioritizing old data?
In most of the products we serve, customers are often interested in recent data in a very granular format ("shallow and wide"), or longer historical queries along a well defined axis ("deep and narrow").
For example, this is where the benefits of TimescaleDB's segmented columnar compression emerges. It optimizes for those queries which are very common in your application, e.g., an IoT application that groups by or selected by deviceID, crypto/fintech analysis based on the ticker symbol, product analytics based on tenantID, etc.
If you look at Clickbench, what most of the queries say are: Scan ALL the data in your database, and GROUP BY one of the 100 columns in the web analytics logs.
- https://github.com/ClickHouse/ClickBench/blob/main/clickhous...
There are almost no time-predicates in the benchmark that Clickhouse created, but perhaps that is not surprising given it was designed for ad-hoc weblog analytics at Yandex.
So yes, Timescale serves many products today that use weather data, but has made different choices than Clickhouse (or things like DuckDB, pg_analytics, etc) to serve those more operational use cases.
-
Variant in Apache Doris 2.1.0: a new data type 8 times faster than JSON for semi-structured data analysis
We tested with 43 Clickbench SQL queries. Queries on the Variant columns are about 10% slower than those on pre-defined static columns, and 8 times faster than those on JSON columns. (For I/O reasons, most cold runs on JSONB data failed with OOM.)
-
Fair Benchmarking Considered Difficult (2018) [pdf]
I have a project dedicated to this topic: https://github.com/ClickHouse/ClickBench
It is important to explain the limitations of a benchmark, provide a methodology, and make it reproducible. It also has to be simple enough, otherwise it will not be realistic to include a large number of participants.
I'm also collecting all database benchmarks I could find: https://github.com/ClickHouse/ClickHouse/issues/22398
- ClickBench – A Benchmark for Analytical DBMS
- FLaNK Stack 05 Feb 2024
-
Why Postgres RDS didn't work for us
Indeed, ClickHouse results were run on an older instance type of the same family and size (c5.4xlarge for ClickHouse and c6a.4xlarge for Timescale), so if anything ClickHouse results are at a slight disadvantage.
This is an open source benchmark - we'd love contributions from Timescale enthusiasts if we missed something: https://github.com/ClickHouse/ClickBench/
-
Show HN: Stanchion – Column-oriented tables in SQLite
Interesting project! Thank you for open sourcing and sharing. Agree that local and embedded analytics are an increasing trend, I see it too.
A couple of questions:
* I’m curious what the difficulties were in the implementation. I suspect it is quite a challenge to implement this support in the current SQLite architecture, and would curious to know which parts were tricky and any design trade-off you were faced with.
* Aside from ease-of-use (install extension, no need for a separate analytical database system), I wonder if there are additional benefits users can anticipate resulting from a single system architecture vs running an embedded OLAP store like DuckDB or clickhouse-local / chdb side-by-side with SQLite? Do you anticipate performance or resource efficiency gains, for instance?
* I am also curious, what the main difficulty with bringing in a separate analytical database is, assuming it natively integrates with SQLite. I may be biased, but I doubt anything can approach the performance of native column-oriented systems, so I'm curious what the tipping point might be for using this extension vs using an embedded OLAP store in practice.
Btw, would love for you or someone in the community to benchmark Stanchion in ClickBench and submit results! (https://github.com/ClickHouse/ClickBench/)
Disclaimer: I work on ClickHouse.
- ClickBench: A Benchmark for Analytical Databases
- DuckDB performance improvements with the latest release
-
DoorDash manages high-availability CockroachDB clusters at scale
interesting. curious if anyone has benchmarked it relative to other dbs. like: https://benchmark.clickhouse.com/
What are some alternatives?
doris - Apache Doris is an easy-to-use, high performance and unified analytics database.
duckdb - DuckDB is an in-process SQL OLAP Database Management System
ClickHouse - ClickHouse® is a free analytics DBMS for big data
TablePlus - TablePlus macOS issue tracker
hosts - 🔒 Consolidating and extending hosts files from several well-curated sources. Optionally pick extensions for porn, social media, and other categories.
clickhouse-bulk - Collects many small inserts to ClickHouse and send in big inserts
LakeSoul - LakeSoul is an end-to-end, realtime and cloud native Lakehouse framework with fast data ingestion, concurrent update and incremental data analytics on cloud storages for both BI and AI applications.
datafusion - Apache DataFusion SQL Query Engine