rqlite
mvsqlite
Our great sponsors
rqlite | mvsqlite | |
---|---|---|
112 | 26 | |
14,760 | 1,293 | |
1.6% | - | |
9.9 | 0.0 | |
7 days ago | 8 days ago | |
Go | Rust | |
MIT License | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
rqlite
-
CursusDB – A new scalable distributed document oriented database
Seems like you could do the same with rqlite [1], since SQLite supports JSON.
[1]: https://rqlite.io
- I'm All-In on Server-Side SQLite
-
So, you want to deploy on the edge?
rqlite[1] creator here, happy to answer any questions. rqlite also supports read-only nodes, which can also help with reads at the "edge". It probably wouldn't scale to 100s of nodes, it is an option.
"rqlite supports adding read-only nodes. You can use this feature to add read scalability to the cluster if you need a high volume of reads, or want to distribute copies of the data nearer to clients – but don’t want those nodes counted towards the quorum. These types of nodes are also known as non-voting nodes."
- LiteFS Cloud: Distributed SQLite with Managed Backups
-
Latest Chat-GPT4 release (May 12th 2023) sure is buggy
result, opErr := db.executeStmtWithConn for the "executeStmt that you , you can manipulate the picture ', you ... i = t given, and it' . If, -212<|endoftext|>
[1] https://rqlite.io/
Yes, it's saved me hours of time -- with certain narrow tasks. For example, it generated these files for me:
https://github.com/rqlite/rqlite/blob/master/auto/backup/sum*
It took a few minutes, might have taken me 1+ hour of some research, and mechanically typing it all in.
-
Ask HN: It's 2023, how do you choose between MySQL and Postgres?
Just to point out, there are now SQLite replication and various "distributed database" projects which seem to work fairly well.
They're probably not as battle tested as the PostgreSQL ones, but they are around, have users, and are actively developed.
The ones I remember off the top of my head:
* https://github.com/rqlite/rqlite <-- more of a "distributed database using RAFT" type of thing
-
rqlite v7.15: the lightweight distributed database built on Go, Raft, and SQLite -- now with automatic backups to S3
Instead of running it as a separate rqlited, integrate it into an existing binary. I answered my own question by looking in https://github.com/rqlite/rqlite/blob/master/cmd/rqlited/main.go
-
Strong Consistency with Raft and SQLite
Is this somehow related to rqlite? https://rqlite.io/
The architecture is very similar.
mvsqlite
-
FoundationDB: A Distributed Key-Value Store
I’ve been using FDB for toy projects for a while. It’s truly rock solid. That being said, I wish there were more layers.
Ideally someone could implement the firestore or dynamodb api on top.
-
Go bindings to SQLite using Wazero
For the rough plan, it's Cloud Backed SQLite meets FoundationDB.
-
SQLite-based databases on the Postgres protocol? Yes we can
- Oh, and if you're wondering about backup to S3, they have that too: https://github.com/libsql/bottomless
- Uh, sqld can integrated with this https://github.com/losfair/mvsqlite, so now your SQLite is backed by FoundationDB!?
- Meanwhile Litestream exists https://github.com/benbjohnson/litestream/
-
We Built Fly Postgres
This was on HN a few months back: https://github.com/losfair/mvsqlite
While not Spanner, it is essentially an open source db like AlloyDB or Aurora, pushing replication and scale out to the storage layer (in this case via FoundationDB). The most interesting bit of mvsqlite is it's multi-writer capabilities, using FoundationDB to perform page-level locks.
I'm neither the creator nor using it in production, but I'd love to see more DBs using FoundationDB as storage. It's a pretty cool solution.
-
Litestream doesn't do SQLite replication anymore (LiteFS does)
Shameless plug of my [mvSQLite](https://github.com/losfair/mvsqlite) project here! It's basically another distributed SQLite, but with support for everything expected from a proper distributed database: synchronous replication, strictly serializable transactions, + scalable reads and writes w/ multiple concurrent writers.
-
SQLite: QEMU All over Again?
The approach to edge-ifying SQLite taken in [1] looks quite promising - using FoundationDb as the storage handles a lot of hairiness.
This project looks really exciting!
I'm working on mvsqlite [1], a distributed SQLite based on FoundationDB. When doing the VFS integration I have always wanted to patch SQLite itself, but didn't because of uncertainty around correctness of the patched version...
A few features on my wishlist:
1. Asynchronous I/O. mvsqlite is currently doing its own prefetch prediction that is not very accurate. I assume higher layers in SQLite have more information that can help with better prediction.
2. Custom page allocator. SQLite internally uses a linked list to manage database pages - this causes contention on any two transactions that both allocate or free pages.
3. Random ROWID, without the `max(int64)` row trick. Sequentially increasing ROWIDs is a primary source of contention, and causes significant INSERT slowdown in my benchmark [2].
-
Show HN: Query SQLite files stored in S3
That DynamoDB VFS looks cool! I agree that the VFS api makes one think about plenty of crazy ideas. Someone is working on a VFS based on Foundation DB[0] that looks very promising. It was recently discussed here[1]
-
Turning SQLite into a Distributed Database
Hi mrkurt!
Litestream/LiteFS are amazing projects. The FUSE-based approach is interesting (I'm implementing something similar in mvSQLite, thanks for the idea!)
> Graceful failure
mvSQLite is designed to continue to operate under degraded network (there is a fault-injection test specifically for checking this property: https://github.com/losfair/mvsqlite/blob/1dd1a80d2ff7263b07a...). Network errors and service unavailability are handled with idempotent retries and not exposed to the application.
> Good for caching
mvSQLite caches pages read and written, and does differential cache invalidation (only remotely modified pages are invalidated in the local page cache). The local cache is just a regular KV store with invalidation strategies, and can be moved onto the disk. So it essentially becomes a consistent local database snapshot.
What are some alternatives?
dqlite - Embeddable, replicated and fault-tolerant SQL engine.
litestream - Streaming replication for SQLite.
cockroach - CockroachDB - the open source, cloud-native distributed SQL database.
bolt
etcd - Distributed reliable key-value store for the most critical data of a distributed system [Moved to: https://github.com/etcd-io/etcd]
TinyGo - Go compiler for small places. Microcontrollers, WebAssembly (WASM/WASI), and command-line tools. Based on LLVM.
go-cache - An in-memory key:value store/cache (similar to Memcached) library for Go, suitable for single-machine applications.
groupcache - groupcache is a caching and cache-filling library, intended as a replacement for memcached in many cases.
budibase - Budibase is an open-source low code platform that helps you build internal tools in minutes 🚀
Bedrock - Rock solid distributed database specializing in active/active automatic failover and WAN replication
noms - The versioned, forkable, syncable database
pgweb - Cross-platform client for PostgreSQL databases