mvsqlite VS datasette-stripe

I’ve been using FDB for toy projects for a while. It’s truly rock solid. That being said, I wish there were more layers.

Ideally someone could implement the firestore or dynamodb api on top.

https://github.com/losfair/mvsqlite

For the rough plan, it's Cloud Backed SQLite meets FoundationDB.

- Oh, and if you're wondering about backup to S3, they have that too: https://github.com/libsql/bottomless

- Uh, sqld can integrated with this https://github.com/losfair/mvsqlite, so now your SQLite is backed by FoundationDB!?

- Meanwhile Litestream exists https://github.com/benbjohnson/litestream/

Shameless plug of my [mvSQLite](https://github.com/losfair/mvsqlite) project here! It's basically another distributed SQLite, but with support for everything expected from a proper distributed database: synchronous replication, strictly serializable transactions, + scalable reads and writes w/ multiple concurrent writers.

This project looks really exciting!

I'm working on mvsqlite [1], a distributed SQLite based on FoundationDB. When doing the VFS integration I have always wanted to patch SQLite itself, but didn't because of uncertainty around correctness of the patched version...

A few features on my wishlist:

1. Asynchronous I/O. mvsqlite is currently doing its own prefetch prediction that is not very accurate. I assume higher layers in SQLite have more information that can help with better prediction.

2. Custom page allocator. SQLite internally uses a linked list to manage database pages - this causes contention on any two transactions that both allocate or free pages.

3. Random ROWID, without the `max(int64)` row trick. Sequentially increasing ROWIDs is a primary source of contention, and causes significant INSERT slowdown in my benchmark [2].

[1] https://github.com/losfair/mvsqlite

[2] https://univalence.me/posts/mvsqlite-bench-20220930

That DynamoDB VFS looks cool! I agree that the VFS api makes one think about plenty of crazy ideas. Someone is working on a VFS based on Foundation DB[0] that looks very promising. It was recently discussed here[1]

[0]: https://github.com/losfair/mvsqlite

[1]: https://news.ycombinator.com/item?id=32269287

If you want to analyse your payment failures with SQL you could use https://table.dog to download them into a SQLite database.

https://table.dog is a CLI that downloads your Stripe account to a SQLite db.

Would appreciate it if you could test it out if you are interested.

https://table.dog (a CLI to download Stripe to SQLite).

I considered Node.js, C#.

I went with Rust for a few reasons:

- Easy to build small portable binaries (as a primary language feature).

- The type checker ensures type consistency when writing out to SQL tables (SQLite is loosely typed). Code that reads from the SQLite database implicitly benefits from Rusts strong type checks.

- Macros to convert structs to SQL insert/updates.

- Reduce the chance of errors at runtime.

- Leverage as much as SQLite's write throughput as possible.

- When converting Stripes Open API JSON spec into Rust code (using another Node.js program), the Rust type checker ensures I have a well formed HTTP client - the strict compiler makes it a good target for generated programs. Read more about this idea at (https://willcrichton.net/notes/rust-the-new-llvm/).

This is a CLI written in Rust to download your Stripe account to a local SQLite database.

It is intended to enable using SQL queries over your data.

There is also a template for fly.io and Datasette to give you a Sigma like web UI:

https://github.com/tabledog/datasette-stripe