sqlite-s3-query VS rqlite

Awesome project!

What about hosting the data in an S3 bucket with "Requestor Pays"? You'd only have the storage cost.

Disables anonymous access (so would a Dropbox share) but reduces your cost massively.

https://docs.aws.amazon.com/AmazonS3/latest/userguide/Reques...

You wouldn't necessarily need an SQL frontend as it's readonly anyway, and there are multiple ways of letting SQLite access databases in S3 buckets, e.g. https://github.com/michalc/sqlite-s3-query

Aha! This is done in https://github.com/uktrade/mbtiles-s3-server, but using an alternative library to query SQLite on S3, https://github.com/michalc/sqlite-s3-query

(Full disclosure: I wrote most of both of these)

I was wondering how this ran queries against SQLite files in an S3 bucket. Here's the answer - it's using some very clever Python ctypes code to implement that HTTP Range header trick from last year:

https://github.com/michalc/sqlite-s3-query/blob/main/sqlite_...

Here's the inspiration for that: https://github.com/phiresky/sql.js-httpvfs

Seems like you could do the same with rqlite [1], since SQLite supports JSON.

[1]: https://rqlite.io

rqlite[1] creator here, happy to answer any questions about rqlite, this latest release, and how it works.

[1] https://rqlite.io

I found simple distributed RQlite https://github.com/rqlite/rqlite based on raft and sqlite. How hard is to add it?

rqlite[1] creator here, happy to answer any questions. rqlite also supports read-only nodes, which can also help with reads at the "edge". It probably wouldn't scale to 100s of nodes, it is an option.

"rqlite supports adding read-only nodes. You can use this feature to add read scalability to the cluster if you need a high volume of reads, or want to distribute copies of the data nearer to clients – but don’t want those nodes counted towards the quorum. These types of nodes are also known as non-voting nodes."

[1] https://rqlite.io/

[2] https://rqlite.io/docs/clustering/read-only-nodes/