duckdb
LevelDB
Our great sponsors
duckdb | LevelDB | |
---|---|---|
51 | 27 | |
15,710 | 34,871 | |
10.4% | 1.2% | |
10.0 | 0.0 | |
5 days ago | about 1 month ago | |
C++ | C++ | |
MIT License | BSD 3-clause "New" or "Revised" License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
duckdb
-
DuckDB performance improvements with the latest release
I'm not sure if the fix is reassuring or not: https://github.com/duckdb/duckdb/pull/9411/files
Just had a look (https://github.com/duckdb/duckdb/issues/9399). Yeah it's worrying that such a trivial query returned incorrect results - but credit to the Devs for getting it fixed quickly.
To my knowledge the only databases that can be described as "military-grade" in terms of testing are SQLite and Postgres.
-
Building a Distributed Data Warehouse Without Data Lakes
It's an interesting question!
The problem is that the data is spread everywhere - no choice about that. So with that in mind, how do you query that data? Today, the idea is that you HAVE to put it into a central location. With tools like Bacalhau[1] and DuckDB [2], you no longer have to - a single query can be sharded amongst all your data - EFFECTIVELY giving you a lot of what you want from a data lake.
It's not a replacement, but if you can do a few of these items WITHOUT moving the data, you will be able to see really significant cost and time savings.
- DuckDB 0.9.0
-
Push or Pull, is this a question?
[4] Switch to Push-Based Execution Model by Mytherin · Pull Request #2393 · duckdb/duckdb (github.com)
-
Show HN: Hydra 1.0 – open-source column-oriented Postgres
it depends on your query obviously.
In general, I did very deep benchmarking of pg, clickhouse and duckdb, and I sure didn't make stupid mistakes like this: https://news.ycombinator.com/item?id=36990831
My dataset has 50B rows and 2tb of data, and I think columnar dbs are very overhiped and I chose pg because:
- pg performance is acceptable, maybe 2-3x times slower than clickhouse and duckdb on some queries if pg is configured correctly and run on compressed storage
- clickhouse and duckdb start falling apart very fast because they specialized on very narrow type of queries: https://github.com/ClickHouse/ClickHouse/issues/47520 https://github.com/ClickHouse/ClickHouse/issues/47521 https://github.com/duckdb/duckdb/discussions/6696
-
🦆 Effortless Data Quality w/duckdb on GitHub ♾️
This action installs duckdb with the version provided in input.
-
Using SQL inside Python pipelines with Duckdb, Glaredb (and others?)
Duckdb: https://github.com/duckdb/duckdb - seems pretty popular, been keeping an eye on this for close to a year now.
-
CSV or Parquet File Format
The Parquet-Go library is very complex, not yet success to use it. So I ask whether DuckDB can provide API https://github.com/duckdb/duckdb/issues/7776
-
DuckDB 0.8.0
Another cool new feature that's not mentioned in the blog post is function chaining:
https://github.com/duckdb/duckdb/pull/6725
I've been using DuckDB for filtering and post-processing data, specially strings, and this will make writing complex queries easier. By combining nested functions[0] and text functions[1], sometimes I don't even need to go into a Python notebook.
LevelDB
-
Codebases to read
I'm partial to how cleanly written https://github.com/google/leveldb is. It is a reasonable size to fully read & grok in not too long.
-
SQLite performance tuning: concurrent reads, multiple GBs and 100k SELECTs/s
Google didnt, thus Chrome started replacing sqlite with https://github.com/google/leveldb
-
Is there a lightweight, stable and embedded database library?
leveldb?
- Ask HN: What's the best source code you've read?
-
LevelDB VS ZoneTree - a user suggested alternative
2 projects | 22 Aug 2022
-
Open Source Databases in Go
goleveldb - Implementation of the LevelDB key/value database in Go.
-
Warp: Lightweight Multi-Key Transactions for Key-Value Stores
I don't know of any rule of English grammar that would lead to this interpretation. If you do, you should immediately write to the maintainers of these websites:
https://redis.com/nosql/key-value-databases/
https://www.mongodb.com/databases/key-value-database
https://aws.amazon.com/nosql/key-value/
https://etcd.io/docs/v3.4/learning/why/
https://riak.com/products/riak-kv/
-
Leveldb File Forensics
https://github.com/google/leveldb -- successfully compiled the tool. This tool seemed the most promising. I tried using the command:
- Ask HN: What are the best key-value self-hosted storage engines?
What are some alternatives?
ClickHouse - ClickHouse® is a free analytics DBMS for big data
RocksDB - A library that provides an embeddable, persistent key-value store for fast storage.
MongoDB - The MongoDB Database
SQLite - Unofficial git mirror of SQLite sources (see link for build instructions)
Redis - Redis is an in-memory database that persists on disk. The data model is key-value, but many different kind of values are supported: Strings, Lists, Sets, Sorted Sets, Hashes, Streams, HyperLogLogs, Bitmaps.
LMDB - Read-only mirror of official repo on openldap.org. Issues and pull requests here are ignored. Use OpenLDAP ITS for issues.
CouchDB - Seamless multi-master syncing database with an intuitive HTTP/JSON API, designed for reliability
sqlite-worker - A simple, and persistent, SQLite database for Web and Workers.
SQLite - Official Git mirror of the SQLite source tree
Apache Cassandra - Mirror of Apache Cassandra
datasette - An open source multi-tool for exploring and publishing data
libmdbx - One of the fastest embeddable key-value ACID database without WAL. libmdbx surpasses the legendary LMDB in terms of reliability, features and performance.