sqld
sqlite-y-crdt
sqld | sqlite-y-crdt | |
---|---|---|
9 | 1 | |
917 | 10 | |
- | - | |
9.6 | 4.5 | |
7 months ago | 4 months ago | |
Rust | Rust | |
MIT License | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
sqld
-
LiteFS Cloud: Distributed SQLite with Managed Backups
There's https://github.com/libsql/sqld , but sqlite's concurrency model doesn't always work well with long-lived transactions (and just the network hop can be slower than a local transaction), especially if you want to write.
-
Get started with libSQL, a next-gen fork of SQLite
For instructions on how to build sqld from source, see the docs.
- Help me sell sqlite to my boss
-
Litestream is awesome for backing up sqlite databases
I also discovered https://github.com/libsql/sqld/ which provides similar functionality ("Bottomless replication"), and exposes the database over the network via the Postgres protocol. I haven't played with this yet, but it would be neat to try running some apps that require Postgres with this.
-
Early impressions of Turso, the edge database from ChiselStrike
There are some language-specific clients suited for data transaction with a Turso database, such as @libsql/client for TypeScript, libsql-client for Python, and libsql-client for Rust(currently works with Cloudflare workers), but for simplicity, we’re going to use curl scripts. We’ll interact with the database by sending HTTP POST requests containing the JSON-encoded SQL queries that we’d otherwise run on the SQL shell as already demonstrated. By supporting database interaction via the HTTP protocol, Turso guarantees easy access from all types of applications, especially edge functions.
-
SQLite-based databases on the Postgres protocol? Yes we can
- And you can embed it in your application and it will really talk to a "SQLite" database over the network: https://github.com/libsql/sqld (not a lot on consistency, I wonder how multiple writers are handled?)
- In 3 months, libsql released WASM functions, S3 WAL and network database sqld
- Sqld – a server mode for libSQL (SQLite fork) using the PostgreSQL wire protocol
- Sqld is a server mode for libSQL
sqlite-y-crdt
-
LiteFS Cloud: Distributed SQLite with Managed Backups
Great that you brought it up. I will fill in the perspective of what I am doing for solving this in Marmot (https://github.com/maxpert/marmot). Today Marmot already records changes via installing triggers to record changes of a table, hence all the offline changes (while Marmot is not running) are never lost. Today when Marmot comes up after a long offline (depending upon max_log_size configuration), it realizes that and tries to catch up changes via restoring a snapshot and then applying rest of logs from NATS (JetStream) change logs. I am working on change that will be publishing those change logs to NATS before it restores snapshots, and once it reapplies those changes after restoring snapshot everyone will have your changes + your DB will be up to date. Now in this case one of the things that bothers people is the fact that if two nodes coming up with conflicting rows the last writer wins.
For that I am also exploring on SQLite-Y-CRDT (https://github.com/maxpert/sqlite-y-crdt) which can help me treat each row as document, and then try to merge them. I personally think CRDT gets harder to reason sometimes, and might not be explainable to an entry level developers. Usually when something is hard to reason and explain, I prefer sticking to simplicity. People IMO will be much more comfortable knowing they can't use auto incrementing IDs for particular tables (because two independent nodes can increment counter to same values) vs here is a magical way to merge that will mess up your data.
What are some alternatives?
litefs-js - JavaScript utilities for working with LiteFS on Fly.io
donutdb - Store and query a sqlite db directly backed by DynamoDB.
mycelite - Mycelite is a SQLite extension that allows you to synchronize changes from one instance of SQLite to another.
litestack
StorX - PHP library for flat-file data storage
StorX-API - A REST API for StorX
marmot - A distributed SQLite replicator built on top of NATS
stream-sqlite - Python function to extract rows from a SQLite file while iterating over its bytes
popcorn-time - Example application using libSQL's sqld + express + Vue.js