SQLite VS dqlite

If you are into alternative storage engines for SQLite, there is also an LSM (Log-Structured Merge-tree) extension in the main repository that is not announced nor documented but seems to work. It’s based on the SQLite 4 project.

https://github.com/sqlite/sqlite/tree/master/ext/lsm1

https://www.charlesleifer.com/blog/lsm-key-value-storage-in-...

The sqlite code base is really well done. Lots of documentation.

https://github.com/sqlite/sqlite

Especially the VM part: https://github.com/spandanb/learndb-py/blob/master/learndb/v...

Compare it with this: https://github.com/sqlite/sqlite/blob/master/src/vdbe.c

That's said, I'm curious how complete this LearnDB is. SQLite is hard to read not only it's old but also it covers a lot of SQL and following SQL spec makes hings complicated. SQLite has great test suite so it's nice if you run the suit against this implementation.

Correct, see the github mirror[1]. I don't know how well supported that feature is compared to main branch. If it was completely stable, then it would have already landed in the main stable branch. Clarity about the roadmap of that branch would be nice.

1. https://github.com/sqlite/sqlite/tree/begin-concurrent

Including comprehensive comments, documentation and tests in a codebase takes time and effort.

Failing to do so creates code that is very difficult to maintain or for someone new to the codebase to understand.

However, time and effort may not be what the organization wants to pay for, and individuals may view their own incomprehensible code as something like job security, as they can't be replaced by someone else easily.

As an example of complicated code that's still well-documented, the open-source sqlite code is a good example, about 1/4 of the B-tree file is comments, every time a variable is defined there's a short note explaining what it's used for, every function has a comment header that's comprehensive, such that someone new to the codebase could construct a map of how it all works fairly quickly. It's a good model for how to avoid the problem:

https://github.com/sqlite/sqlite/blob/master/src/btree.c

If you're interested in this, here are some related projects that all take slightly different approaches:

- LiteSync directly competes with Marmot and supports DDL sync, but is closed source commercial (similar to SQLite EE): https://litesync.io

- dqlite is Canonical's distributed SQLite that depends on c-raft and kernel-level async I/O: https://dqlite.io

- cr-sqlite is a Rust-based loadable extension that adds CRDT changeset generation and reconciliation to SQLite: https://github.com/vlcn-io/cr-sqlite

Slightly related but not really (no multi writer, no C-level SQLite API or other restrictions):

- comdb2 (Bloombergs multi-homed RDMS using SQLite as the frontend)

- rqlite: RDMS with HTTP API and SQLite as the storage engine, used for replication and strong consistency (does not scale writes)

- litestream/LiteFS: disaster recovery replication

- liteserver: active read-only replication (predecessor of LiteSync)

I'd be curious for a similar tuning with Dqlite: https://github.com/canonical/dqlite

Anyone knows how the DB this is about, https://rqlite.io/, compares with https://dqlite.io/ by Canonical (both seem to be distributed versions of sqlite)?

For folks' context, the new tool that's being discussed in the thread mentioned by the parent here is litefs [0], as well as which you can also look at rqlite [1] and dqlite [2], which all provide different trade-offs (e.g. rqlite is 'more strongly consistent' than litefs).

[0]: https://github.com/superfly/litefs

[1]: https://github.com/rqlite/rqlite

[2]: https://github.com/canonical/dqlite

I presume you're familiar with https://github.com/canonical/dqlite (made by my employer) and https://github.com/rqlite/rqlite (unrelated)? How will mvsqlite compare to those?

Check out dqlite, it's sqlite but with a raft consensus to distribute changes through a log: https://dqlite.io/ You can link it in as a library too, it sounds like exactly what you want.