verneuil
mvsqlite
Our great sponsors
verneuil | mvsqlite | |
---|---|---|
5 | 26 | |
389 | 1,315 | |
0.8% | - | |
6.7 | 0.0 | |
about 2 months ago | 1 day ago | |
C | Rust | |
MIT License | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
verneuil
- Show HN: Query SQLite files stored in S3
-
Embedded database with VFS support?
It'd be process wide. If you want an example can check out the example using a vfs here. There's an explicit passing of vfs there and an implicit usage of it. https://github.com/backtrace-labs/verneuil/blob/main/examples/rusqlite_integration.rs
- LiteFS a FUSE-based file system for replicating SQLite
-
A database for 2022 · Tailscale
It doesn't even have to be WAL-based system. Backtrace Labs has a SQLite virtual file system (VFS) called Verneuil that works similarly but works with the rollback journal instead of the WAL.
-
Ask HN: P2P Databases?
https://github.com/backtrace-labs/verneuil/ is one way to address the diffing / read replica part of the problem. I believe it's compatible with gossipping: most of the data is in small content-addressed chunks, with small manifests that tell clients what chunks to fetch and how to reassemble them to recreate a sqlite database. There's already client-side caching to persistent storage, and chunks can be fetched on demand.
Sharing replication data P2P, while retaining the simplicity of a single authoritative writer per database, is explicitly part of the project's long-term goals!
mvsqlite
-
FoundationDB: A Distributed Key-Value Store
I’ve been using FDB for toy projects for a while. It’s truly rock solid. That being said, I wish there were more layers.
Ideally someone could implement the firestore or dynamodb api on top.
-
Go bindings to SQLite using Wazero
For the rough plan, it's Cloud Backed SQLite meets FoundationDB.
-
SQLite-based databases on the Postgres protocol? Yes we can
- Oh, and if you're wondering about backup to S3, they have that too: https://github.com/libsql/bottomless
- Uh, sqld can integrated with this https://github.com/losfair/mvsqlite, so now your SQLite is backed by FoundationDB!?
- Meanwhile Litestream exists https://github.com/benbjohnson/litestream/
- We Built Fly Postgres
-
Litestream doesn't do SQLite replication anymore (LiteFS does)
Shameless plug of my [mvSQLite](https://github.com/losfair/mvsqlite) project here! It's basically another distributed SQLite, but with support for everything expected from a proper distributed database: synchronous replication, strictly serializable transactions, + scalable reads and writes w/ multiple concurrent writers.
-
SQLite: QEMU All over Again?
This project looks really exciting!
I'm working on mvsqlite [1], a distributed SQLite based on FoundationDB. When doing the VFS integration I have always wanted to patch SQLite itself, but didn't because of uncertainty around correctness of the patched version...
A few features on my wishlist:
1. Asynchronous I/O. mvsqlite is currently doing its own prefetch prediction that is not very accurate. I assume higher layers in SQLite have more information that can help with better prediction.
2. Custom page allocator. SQLite internally uses a linked list to manage database pages - this causes contention on any two transactions that both allocate or free pages.
3. Random ROWID, without the `max(int64)` row trick. Sequentially increasing ROWIDs is a primary source of contention, and causes significant INSERT slowdown in my benchmark [2].
- Show HN: mvSQLite v0.2
- mvsqlite: Distributed SQLite built on FoundationDB
-
Show HN: Query SQLite files stored in S3
That DynamoDB VFS looks cool! I agree that the VFS api makes one think about plenty of crazy ideas. Someone is working on a VFS based on Foundation DB[0] that looks very promising. It was recently discussed here[1]
- GitHub - losfair/mvsqlite: Distributed, MVCC SQLite that runs on FoundationDB.
What are some alternatives?
litefs - FUSE-based file system for replicating SQLite databases across a cluster of machines
awesome-sqlite - A curated list of awesome things related to SQLite
go-ds-crdt - A distributed go-datastore implementation using Merkle-CRDTs.
dqlite - Embeddable, replicated and fault-tolerant SQL engine.
WCDB - WCDB is a cross-platform database framework developed by WeChat.
rqlite - The lightweight, distributed relational database built on SQLite.
bb-remote-execution - Tools for Buildbarn to allow remote execution of build actions
datasette-stripe - A web SQL interface to your Stripe account using Datasette.
s3fs - S3 Filesystem
blueboat - All-in-one, multi-tenant serverless JavaScript runtime.