starrocks VS doris

tidb has been around for a while, it is distributed, written in Go and Rust, and MySQL compatible. https://github.com/pingcap/tidb

Somewhat relatedly, StarRocks is also MySQL compatible, written in Java and C++, but it's tackling OLAP use-cases. https://github.com/StarRocks/starrocks

I think you're talking about doing denormalization before importing data into an OLAP system to avoid subsequent joins. However, this greatly limits the flexibility of data modeling. Moreover, denormalization can be a headache-inducing process. In fact, I have tested StarRocks (https://github.com/StarRocks/starrocks), and it is capable of performing joins while streaming data imports, and the speed is very fast. It's worth giving it a try.

https://github.com/apache/doris Is a great example. Same for it's cousin https://github.com/StarRocks/starrocks that was an early fork of the doris project.

To be fair, these are the only examples I can think of and I only learned of these as I'm standing up new data infra using starrocks.

ClickHouseClickHouse and Starrocks are similar. They are both columnar databases powered by vectorization tech, which means they are really fast.

StarRocks, it’s a Linux Foundation project now, but a lot of the initial team and community behind it came from China.

https://github.com/StarRocks/starrocks

Funny that I hadn’t heard of them in the database space till they showed up at the top of ClickBench. Makes me wonder what other interesting projects I’m missing out on in China.

Full disclosure - I work for StarRocks (starrocks.com)

First of all, this is great. Transparent and healthy competition is always great for the customers!

Regarding the joined table queries that are missing in the tests, this is exactly why we built StarRocks - to give people the best performance of complex analytics queries on both joined tables and single tables.

I encourage you to checkout this blog: https://starrocks.medium.com/starrocks-outperforms-clickhous...

And, give us a star if you think we are doing the right thing: https://github.com/StarRocks/starrocks

Follow us on LinkedIn for the latest updates: https://www.linkedin.com/company/starrocks

As an open-source real-time data warehouse, Apache Doris provides semi-structured data processing capabilities, and the newly-released version 2.1.0 makes a stride in this direction. Before V2.1, Apache Doris stores semi-structured data as JSON files. However, during query execution, the real-time parsing of JSON data leads to high CPU and I/O consumption in addition to high query latency, especially when the dataset is huge and complicated. Moreover, the lack of a pre-defined schema means there is no handle for query optimization.

Apache Doris is a real-time data warehouse.

Learn more about Apache Doris or find the Doris makers on Slack.

As you can imagine, a long and complicated data pipeline is high-maintenance and detrimental to development efficiency. Moreover, they are not capable of ad-hoc queries. So as an upgrade to our data warehouse, we replaced most of these components with Apache Doris, a unified analytic database.

GitHub source code: https://github.com/apache/doris/tree/branch-2.0

The key to Architecture 3.0 is the combination of Flink and Doris, so this is how to connect them. Probably the most important code in building architecture 3. flink-demo stream-load-demo

https://github.com/apache/doris Is a great example. Same for it's cousin https://github.com/StarRocks/starrocks that was an early fork of the doris project.

To be fair, these are the only examples I can think of and I only learned of these as I'm standing up new data infra using starrocks.

We optimized Apache Doris to solve these problems. (Pull Request on Github)

So, in Storage Architecture 2.0, we introduced Apache Doris and Apache Spark. The whole data pipeline was a Y-shaped diagram.