chdb
hydra
chdb | hydra | |
---|---|---|
18 | 26 | |
1,726 | 2,647 | |
4.8% | 4.1% | |
9.5 | 8.5 | |
about 24 hours ago | 9 days ago | |
C++ | C | |
Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
chdb
- FLaNK Stack Weekly 06 Nov 2023
-
DB Pilot: Query Postgres, files, S3 and more – all at once, from your laptop
Hey HN, creator of DB Pilot here.
I first announced DB Pilot on HN back in April: https://news.ycombinator.com/item?id=35761979.
Since then a lot has improved: More databases are supported, most of the product can now be used for free, and most importantly:
The app now comes with an analytics workspace powered by an embedded ClickHouse instance, running locally on your machine. This allows you to query local files, files on S3, PostgreSQL, SQLite & more - and all of those at once.
Embedding ClickHouse was possible thanks to chDB (https://github.com/chdb-io/chdb). A recent discussion on HN about it: https://news.ycombinator.com/item?id=37985005
- ChDB: Embedded OLAP SQL Engine Powered by ClickHouse
-
DuckDB 0.9.0
I recommend using ClickHouse instead of DuckDB.
It has been around since 2016, and it covers and extends the feature set of DuckDB with a huge margin. Worth noting that it never has breaking changes in its table format MergeTree.
I'm tracking the progress of DuckDB and see that it is modeled after ClickHouse, but does not approach it in terms of feature completeness, stability, or performance.
The closest to DuckDB option is to use its self-contained version, clickhouse-local: https://clickhouse.com/blog/extracting-converting-querying-l... or an embedded version, chdb: https://github.com/chdb-io/chdb
-
Is ClickHouse Moving Away from Open Source?
Different beasts, but if by any chance you love ClickHouse already and just want to run OLAP queries in-process, there's chdb: https://github.com/chdb-io/chdb
- ChDB: An Embedded OLAP SQL Engine Powered by ClickHouse
-
PRQL, Pipelined Relational Query Language
> Can you embed it in Python as a library?
https://github.com/chdb-io/chdb
pip install chdb
-
Using SQL inside Python pipelines with Duckdb, Glaredb (and others?)
New kid on the block that I prefer over DuckDB is CHDB (https://github.com/chdb-io/chdb). Embedded ClickHouse, so once you out grow your laptop you can simply move to an actual OLAP that's Open-source.
- ClickHouse-local and chdb performance issue on clickbench Q.23 Q28
hydra
-
Using ClickHouse to scale an events engine
Don't feel bad, lots of people get bitten by not reading all the way down to the bottom of their readme: https://github.com/hydradatabase/hydra/blob/v1.1.2/README.md... While Hydra may very well license their own code Apache 2, they ship the AGPLv3 columnar which to my very best IANAL understanding taints the whole stack and AGPLv3's everything all the way through https://github.com/hydradatabase/hydra/blob/v1.1.2/columnar/...
-
Moving a Billion Postgres Rows on a $100 Budget
Columnar store PostgreSQL extension exists, here are two but I think I’m missing at least another one:
https://github.com/citusdata/cstore_fdw
https://github.com/hydradatabase/hydra
You can also connect other stores using the foreign data wrappers, like parquet files stored on an object store, duckdb, clickhouse… though the joins aren’t optimised as PostgreSQL would do full scan on the external table when joining.
- Hydra (YC W22) adds upsert to columnar Postgres
- Hydra
-
Is ClickHouse Moving Away from Open Source?
New column store alternative : https://github.com/hydradatabase/hydra
HN: https://news.ycombinator.com/item?id=37571974
-
Show HN: Hydra - Open-Source Columnar Postgres
some previous discussions:
https://news.ycombinator.com/item?id=37247945
https://news.ycombinator.com/item?id=36987920
and a relevant observation is that there are actually multiple license files in the repo so the consumer should read their explicit licensing section of the readme <https://github.com/hydradatabase/hydra#license> since the GitHub sidebar is misleading
-
CDC from postgres to postgres.
Hydra DB Link to Github -> Worked well for aggregated query usecases but not for queries that build reports. Also, data insertion and updation is abyssmal on columnar dbs.
-
How Query Engines Work
There's a lot of experience about db operation and how to approach MVCC encoded in PostgreSQL that shouldn't be underestimated.
[0]: https://github.com/hydradatabase/hydra
-
Hydra: Column-Oriented Postgres
And just like last time, watch out for the misleading GitHub license detector because it's not entirely Apache as the GitHub summary claims but rather *some* is Apache and buried in the interior is some AGPL stuff: https://github.com/hydradatabase/hydra#license
What are some alternatives?
risingwave - SQL stream processing, analytics, and management. PostgreSQL simplicity, unrivaled performance, and seamless elasticity. 🚀 10x more productive. 🚀 10x more cost-efficient.
duckdb - DuckDB is an in-process SQL OLAP Database Management System
openvino_notebooks - 📚 Jupyter notebook tutorials for OpenVINO™
citus - Distributed PostgreSQL as an extension
duckdb-wasm - WebAssembly version of DuckDB
ClickHouse - ClickHouse® is a free analytics DBMS for big data
chdb-cli - Simple CLI / REPL for chdb made in Python
postgres - PostgreSQL in Neon
sqlite_blaster_python - A library for creating huge Sqlite indexes at breakneck speeds
Udacity-Data-Engineering-Projects - Few projects related to Data Engineering including Data Modeling, Infrastructure setup on cloud, Data Warehousing and Data Lake development.
glaredb - GlareDB: An analytics DBMS for distributed data
vasco - vasco: MIC & MINE statistics for Postgres