LevelDB is a fast key-value storage library written at Google that provides an ordered mapping from string keys to string values.
Google didnt, thus Chrome started replacing sqlite with https://github.com/google/leveldb
Some bunch of test scripts to generate a SQLite DB with 1B rows in fastest possible way
I am experimenting with SQLite, where I try inserting 1B rows in under a minute. The current best is inserting 100M rows at 23s. I cut many corners to get performance, but the tweaks might suit your workload.
I have explained my rationale and approach here - https://avi.im/blag/2021/fast-sqlite-inserts/
the repo link - https://github.com/avinassh/fast-sqlite3-inserts
Collect and Analyze Billions of Data Points in Real Time. Manage all types of time series data in a single, purpose-built database. Run at any scale in any environment in the cloud, on-premises, or at the edge.
Embeddable, replicated and fault tolerant SQL engine.
I'd be curious for a similar tuning with Dqlite: https://github.com/canonical/dqlite
Flexible tool to build planet-scale vector tilesets from OpenStreetMap data fast
I spent a while optimizing sqlite inserts for planetiler, this is what I came up with:
It batches inserts into bulk statements and is able to do writes in the 500k+ per second range, and reads are 300-400k/s using those settings.
Strong Consistency with Raft and SQLite
6 projects | news.ycombinator.com | 21 Mar 2023
9 years of open-source database development: reviewing the designs
2 projects | /r/programming | 10 Mar 2023
Cloud database for tomorrow's applications (written in Rust)
7 projects | news.ycombinator.com | 22 Aug 2022
1 project | news.ycombinator.com | 25 Apr 2022
Ask HN: 你是否使用过SQLite作为主数据库？ (Ask HN: Have you used SQLite as a primary database?)
1 project | /r/hnzh | 25 Apr 2022