sqllogictest
citus
sqllogictest | citus | |
---|---|---|
1 | 61 | |
35 | 9,903 | |
- | 1.8% | |
5.7 | 9.4 | |
4 months ago | 4 days ago | |
C | C | |
- | GNU Affero General Public License v3.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
sqllogictest
-
Goal: Pass all 4259065 tests in sqllogictest in 1 week
That specific SQLite test suite (one of four [1]) has loads of generated SQL functions, and a long tail of more complex hand-written tests. That 95% of the test suite will be mostly generated queries that follow the same basic pattern of joins and projections with basic arithmetic and comparisons. See for example [2] and [3].
The generated tests are not designed to test a wide breadth of features of the SQL language, and passing them with a simple engine is very doable. A lot of the value of these tests is that the sheer volume of queries tends to find obscure problems in optimizers that would not easily surface otherwise. That is of course not a problem in a simple engine that does not have an optimizer.
[1] https://www.sqlite.org/testing.html#test_harnesses
[2] https://github.com/gregrahn/sqllogictest/blob/master/test/ra...
[3] https://raw.githubusercontent.com/gregrahn/sqllogictest/mast...
citus
- SPQR 1.3.0: a production-ready system for horizontal scaling of PostgreSQL
- Citus: PostgreSQL extension that transforms Postgres into a distributed database
-
Figma's Databases team lived to tell the scale
I see they don't mention Citus (https://github.com/citusdata/citus), which is already a fairly mature native Postgres extension. From the details given in the article, in sounds like they just reimplemented it.
I wonder if they were unaware of it or disregarded it for a reason —I currently am in a similar situation as the one described in the blog, trying to shard a massive Postgres DB.
-
PostgreSQL Is Enough
It is possible, if you pay for it. You can do Multi-AZ Clustered Instances in RDS, where you get the benefits of Multi-AZ failover with traffic sharing.
If you can run your own infra – at least on an EC2 level – you can do things like Citus [0] for Postgres, which is about as close to "just add database nodes" as you'll get.
[0]: https://www.citusdata.com/
-
Vitess 18
So while searching for something like this for postgres I came across citus. Any one know how that stacks up?
https://github.com/citusdata/citus
- In-Depth Guide: Citus Technical Readme
-
Revolutionizing Database Scaling with CitusDB
References: CitusDB
- Squeeze the hell out of the system you have
- Show HN: Hydra 1.0 – open-source column-oriented Postgres
- Schema-based sharding comes to PostgreSQL with Citus
What are some alternatives?
TDengine - TDengine is an open source, high-performance, cloud native time-series database optimized for Internet of Things (IoT), Connected Cars, Industrial IoT and DevOps.
Greenplum - Greenplum Database - Massively Parallel PostgreSQL for Analytics. An open-source massively parallel data platform for analytics, machine learning and AI.
hytradboi-jam-2022
yugabyte-db - YugabyteDB - the cloud native distributed SQL database for mission-critical applications.
TimescaleDB - An open-source time-series SQL database optimized for fast ingest and complex queries. Packaged as a PostgreSQL extension.
vitess - Vitess is a database clustering system for horizontal scaling of MySQL.
scc - Sloc, Cloc and Code: scc is a very fast accurate code counter with complexity calculations and COCOMO estimates written in pure Go
zig - General-purpose programming language and toolchain for maintaining robust, optimal, and reusable software.
dbt-core - dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.
stolon - PostgreSQL cloud native High Availability and more.
pg_auto_failover - Postgres extension and service for automated failover and high-availability