Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality. Learn more →
Top 23 distributed-database Open-Source Projects
-
-
tidb
TiDB is an open-source, cloud-native, distributed, MySQL-Compatible database for elastic scale and real-time analytics. Try AI-powered Chat2Query free at : https://tidbcloud.com/free-trial
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
-
-
-
shardingsphere
Distributed SQL transaction & query engine for data sharding, scaling, encryption, and more - on any database.
-
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
-
ArangoDB
🥑 ArangoDB is a native multi-model database with flexible data models for documents, graphs, and key-values. Build high performance applications using a convenient SQL-like query language or JavaScript extensions.
-
-
-
-
Trino
Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)
-
yugabyte-db
YugabyteDB - the cloud native distributed SQL database for mission-critical applications.
-
starrocks
StarRocks, a Linux Foundation project, is a next-generation sub-second MPP OLAP database for full analytics scenarios, including multi-dimensional analytics, real-time analytics, and ad-hoc queries. InfoWorld’s 2023 BOSSIE Award for best open source software.
-
oceanbase
OceanBase is an enterprise distributed relational database with high availability, high performance, horizontal scalability, and compatibility with SQL standards.
-
risingwave
Cloud-native SQL stream processing, analytics, and management. KsqlDB and Apache Flink alternative. 🚀 10x more productive. 🚀 10x more cost-efficient.
-
-
Crate
CrateDB is a distributed and scalable SQL database for storing and analyzing massive amounts of data in near real-time, even with complex queries. It is PostgreSQL-compatible, and based on Lucene.
-
awesome-blockchains
A collection about awesome blockchains - open distributed public databases w/ crypto hashes incl. git ;-). Blockchains are the new tulips :tulip::tulip::tulip:. Distributed is the new centralized.
-
ydb
YDB is an open source Distributed SQL Database that combines high availability and scalability with strong consistency and ACID transactions
-
Olric
Distributed in-memory object store. It can be used as an embedded Go library and a language-independent service.
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
Project mention: Oracle Linux 8.8'de PostgreSQL 13 Yedekli Yapı Nasıl Kurulur? - Patroni, ETCD, HAProxy | dev.to | 2023-12-07sudo dnf -y install curl wget vim ETCD_RELEASE=$(curl -s https://api.github.com/repos/etcd-io/etcd/releases/latest|grep tag_name | cut -d '"' -f 4) echo $ETCD_RELEASE wget https://github.com/etcd-io/etcd/releases/download/${ETCD_RELEASE}/etcd-${ETCD_RELEASE}-linux-amd64.tar.gz tar xvf etcd-${ETCD_RELEASE}-linux-amd64.tar.gz cd etcd-${ETCD_RELEASE}-linux-amd64 sudo mv etcd* /usr/local/bin ls /usr/local/bin /usr/local/bin/etcd --version
Project mention: A MySQL compatible database engine written in pure Go | news.ycombinator.com | 2024-04-09tidb has been around for a while, it is distributed, written in Go and Rust, and MySQL compatible. https://github.com/pingcap/tidb
Somewhat relatedly, StarRocks is also MySQL compatible, written in Java and C++, but it's tackling OLAP use-cases. https://github.com/StarRocks/starrocks
Project mention: We Built a 19 PiB Logging Platform with ClickHouse and Saved Millions | news.ycombinator.com | 2024-04-02Yes, we are working on it! :) Taking some of the learnings from current experimental JSON Object datatype, we are now working on what will become the production-ready implementation. Details here: https://github.com/ClickHouse/ClickHouse/issues/54864
Variant datatype is already available as experimental in 24.1, Dynamic datatype is WIP (PR almost ready), and JSON datatype is next up. Check out the latest comment on that issue with how the Dynamic datatype will work: https://github.com/ClickHouse/ClickHouse/issues/54864#issuec...
CockroachDB is an open source distributed SQL database designed for scalability and resilience. While it offers SQL databases, CockroachDB is also compatible with PostgreSQL.
Project mention: Show HN: I made a tool to easily compare pricing of developer tools and services | news.ycombinator.com | 2024-04-23you should add https://surrealdb.com -- basically an open source firebase. and they will launch a paid cloud offering soon.
Opposite to what the documentation tells, the full prefix is jdbc:shardingsphere:absolutepath. I've opened a PR to fix the documentation.
Project mention: The lightweight, easy-to-use, distributed relational database built on SQLite | news.ycombinator.com | 2024-02-23
Actually, Apple does this for iCloud! They use FoundationDB[1] to store billions of databases, one for each user (plus shared or global databases).
See: https://read.engineerscodex.com/p/how-apple-built-icloud-to-...
Discussed on HN at the time: https://news.ycombinator.com/item?id=39028672
[1]: https://github.com/apple/foundationdb https://en.wikipedia.org/wiki/FoundationDB
Project mention: Ask HN: When is pure functional programming beneficial? | news.ycombinator.com | 2023-07-11... or working in an environment or on a problem for which functional patterns apply.
Suppose you are writing a "CRUD" app that writes to a relational database, how do you apply functional programming to that? The whole point of an application like that is that it makes side effects.
In some cases you can break those problems down into functional pieces. Consider Python drivers for a product like
https://www.arangodb.com/
One major problem is that you want drivers that work synchronously and asynchronously, the structure of the average api call is something like
def query(parameters):
Apache ZooKeeper — a distributed coordination, synchronization, and configuration service (written in Java);
Project mention: SPQR 1.3.0: a production-ready system for horizontal scaling of PostgreSQL | news.ycombinator.com | 2024-03-25
Project mention: Need insights to build a distributed key value store from scratch. | /r/DistributedComputing | 2023-12-08Pls check this course: https://github.com/pingcap/talent-plan . It includes how to implement sql and key value store. It’s an awesome course
Project mention: Trino: Fast distributed SQL query engine for big data analytics | news.ycombinator.com | 2024-03-19
Project mention: Best Practice: use the same datatypes for comparisons, like joins and foreign keys | dev.to | 2024-02-01It is possible to apply Batched Nested Loop but with additional code that checks the range of the outer bigint and compare it only if it matches the range of integer. This has been added in YugabyteDB 2.21 with #20715 YSQL: Allow BNL on joins over different integer types to help migrations from PostgreSQL with such datatype inconsistencies.
Project mention: A MySQL compatible database engine written in pure Go | news.ycombinator.com | 2024-04-09tidb has been around for a while, it is distributed, written in Go and Rust, and MySQL compatible. https://github.com/pingcap/tidb
Somewhat relatedly, StarRocks is also MySQL compatible, written in Java and C++, but it's tackling OLAP use-cases. https://github.com/StarRocks/starrocks
Project mention: Show HN: OceanBase – An open-source distributed SQL database written in C++ | news.ycombinator.com | 2023-05-23
Project mention: Proton, a fast and lightweight alternative to Apache Flink | news.ycombinator.com | 2024-01-30How does this compare to RisingWave and Materialize?
https://github.com/risingwavelabs/risingwave
There https://ydb.tech/ open source db that uses erasure coding for replication in single zone/region.
Project mention: Olric: Distributed, embeddable in-memory data structures in Go | news.ycombinator.com | 2024-02-05
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
distributed-database related posts
- Show HN: I made a tool to easily compare pricing of developer tools and services
- A MySQL compatible database engine written in pure Go
- Figma's Databases team lived to tell the scale
- Best Practice: use the same datatypes for comparisons, like joins and foreign keys
- Live Queries in Rust
- Jonathan Katz: Thoughts on PostgreSQL in 2024
- Maybe Getting Rid of Your QA Team Was Bad
-
A note from our sponsor - InfluxDB
www.influxdata.com | 26 Apr 2024
Index
What are some of the best open-source distributed-database projects? This list will help you:
| Project | Stars | |
|---|---|---|
| 1 | etcd | 46,345 |
| 2 | tidb | 36,096 |
| 3 | ClickHouse | 34,153 |
| 4 | cockroach | 29,076 |
| 5 | surrealdb | 25,191 |
| 6 | shardingsphere | 19,425 |
| 7 | rqlite | 14,862 |
| 8 | foundationdb | 13,971 |
| 9 | ArangoDB | 13,340 |
| 10 | awesome-bigdata | 12,792 |
| 11 | Apache ZooKeeper | 11,925 |
| 12 | citus | 9,801 |
| 13 | talent-plan | 9,800 |
| 14 | Trino | 9,552 |
| 15 | yugabyte-db | 8,486 |
| 16 | starrocks | 7,764 |
| 17 | oceanbase | 7,402 |
| 18 | risingwave | 6,283 |
| 19 | dynomite | 4,161 |
| 20 | Crate | 3,955 |
| 21 | awesome-blockchains | 3,694 |
| 22 | ydb | 3,409 |
| 23 | Olric | 3,005 |
Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com