sqld
postlite
sqld | postlite | |
---|---|---|
9 | 18 | |
917 | 1,190 | |
- | - | |
9.6 | 0.0 | |
7 months ago | 7 months ago | |
Rust | Go | |
MIT License | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
sqld
-
LiteFS Cloud: Distributed SQLite with Managed Backups
There's https://github.com/libsql/sqld , but sqlite's concurrency model doesn't always work well with long-lived transactions (and just the network hop can be slower than a local transaction), especially if you want to write.
-
Get started with libSQL, a next-gen fork of SQLite
For instructions on how to build sqld from source, see the docs.
- Help me sell sqlite to my boss
-
Litestream is awesome for backing up sqlite databases
I also discovered https://github.com/libsql/sqld/ which provides similar functionality ("Bottomless replication"), and exposes the database over the network via the Postgres protocol. I haven't played with this yet, but it would be neat to try running some apps that require Postgres with this.
-
Early impressions of Turso, the edge database from ChiselStrike
There are some language-specific clients suited for data transaction with a Turso database, such as @libsql/client for TypeScript, libsql-client for Python, and libsql-client for Rust(currently works with Cloudflare workers), but for simplicity, we’re going to use curl scripts. We’ll interact with the database by sending HTTP POST requests containing the JSON-encoded SQL queries that we’d otherwise run on the SQL shell as already demonstrated. By supporting database interaction via the HTTP protocol, Turso guarantees easy access from all types of applications, especially edge functions.
-
SQLite-based databases on the Postgres protocol? Yes we can
- And you can embed it in your application and it will really talk to a "SQLite" database over the network: https://github.com/libsql/sqld (not a lot on consistency, I wonder how multiple writers are handled?)
- In 3 months, libsql released WASM functions, S3 WAL and network database sqld
- Sqld – a server mode for libSQL (SQLite fork) using the PostgreSQL wire protocol
- Sqld is a server mode for libSQL
postlite
-
SQLedge: Replicate Postgres to SQLite on the Edge
#. SQLite WAL mode
From https://www.sqlite.org/isolation.html https://news.ycombinator.com/item?id=32247085 :
> [sqlite] WAL mode permits simultaneous readers and writers. It can do this because changes do not overwrite the original database file, but rather go into the separate write-ahead log file. That means that readers can continue to read the old, original, unaltered content from the original database file at the same time that the writer is appending to the write-ahead log
#. superfly/litefs: aFUSE-based file system for replicating SQLite https://github.com/superfly/litefs
#. sqldiff: https://www.sqlite.org/sqldiff.html https://news.ycombinator.com/item?id=31265005
#. dolthub/dolt: https://github.com/dolthub/dolt
> Dolt can be set up as a replica of your existing MySQL or MariaDB database using standard MySQL binlog replication. Every write becomes a Dolt commit. This is a great way to get the version control benefits of Dolt and keep an existing MySQL or MariaDB database.
#. pganalyze/libpg_query: https://github.com/pganalyze/libpg_query :
> C library for accessing the PostgreSQL parser outside of the server environment
#. Ibis + Substrait [ + DuckDB ]
> ibis strives to provide a consistent interface for interacting with a multitude of different analytical execution engines, most of which (but not all) speak some dialect of SQL.
> Today, Ibis accomplishes this with a lot of help from `sqlalchemy` and `sqlglot` to handle differences in dialect, or we interact directly with available Python bindings (for instance with the pandas, datafusion, and polars backends).
> [...] `Substrait` is a new cross-language serialization format for communicating (among other things) query plans. It's still in its early days, but there is already nascent support for Substrait in Apache Arrow, DuckDB, and Velox.
#. benbjohnson/postlite: https://github.com/benbjohnson/postlite
> postlite is a network proxy to allow access to remote SQLite databases over the Postgres wire protocol. This allows GUI tools to be used on remote SQLite databases which can make administration easier.
> The proxy works by translating Postgres frontend wire messages into SQLite transactions and converting results back into Postgres response wire messages. Many Postgres clients also inspect the pg_catalog to determine system information so Postlite mirrors this catalog by using an attached in-memory database with virtual tables. The proxy also performs minor rewriting on these system queries to convert them to usable SQLite syntax.
> Note: This software is in alpha. Please report bugs. Postlite doesn't alter your database unless you issue INSERT, UPDATE, DELETE commands so it's probably safe. If anything, the Postlite process may die but it shouldn't affect your database.
#. > "Hosting SQLite Databases on GitHub Pages" (2021) re: sql.js-httpvfs, DuckDB https://news.ycombinator.com/item?id=28021766
#. awesome-db-tools https://github.com/mgramin/awesome-db-tools
-
SQLite-based databases on the postgres protocol? Yes we can!
Ben Johnson poked around in this space last year too https://github.com/benbjohnson/postlite
-
SQLite-based databases on the Postgres protocol? Yes we can
Note that this already exists on top of SQLite proper - authored by Ben Johnson (Litestream, Fly.io etc.) - https://github.com/benbjohnson/postlite
- Hctree is an experimental high-concurrency database back end for SQLite
-
WAL Mode in LiteFS
Currently, you need to SSH in and use the sqlite3 CLI on the server. There has been some work in this area but it's all still rough around the edges. I wrote a server called Postlite[1] that exposes remote SQLite databases over the Postgres wire protocol but it's very alpha. :)
I'd love to see more work in this area. Ricardo Ander-Egg wrote a remote management tool called litexplore[2] that connects over SSH to the SQLite CLI behind the scenes. I haven't used it but I think there's a lot of potential with that approach.
[1]: https://github.com/benbjohnson/postlite
[2]: https://github.com/litements/litexplore
-
Go and SQLite in the Cloud
I've not use this myself, but Ben Johnson's https://github.com/benbjohnson/postlite in front of SQlite might allow you to use PostgREST? I recall him saying on a podcast that his goal was to be able to point the large ecosystem of PG tools at SQlite.
- GitHub - benbjohnson/postlite: Postgres wire compatible SQLite proxy.
- Postgres wire兼容的SQLite代理 (Postgres wire compatible SQLite proxy)
- Postgres wire compatible SQLite proxy
- postlite: Postgres wire compatible SQLite proxy
What are some alternatives?
litefs-js - JavaScript utilities for working with LiteFS on Fly.io
sqlitebrowser - Official home of the DB Browser for SQLite (DB4S) project. Previously known as "SQLite Database Browser" and "Database Browser for SQLite". Website at:
mycelite - Mycelite is a SQLite extension that allows you to synchronize changes from one instance of SQLite to another.
tuql - Automatically create a GraphQL server from a SQLite database or a SQL file
StorX - PHP library for flat-file data storage
marmot - A distributed SQLite replicator built on top of NATS
StorX-API - A REST API for StorX
sshfs - A network filesystem client to connect to SSH servers
stream-sqlite - Python function to extract rows from a SQLite file while iterating over its bytes
Apache Calcite - Apache Calcite
donutdb - Store and query a sqlite db directly backed by DynamoDB.
awesome-graphql - Awesome list of GraphQL