workers-wasi VS rusqlite

The examples seemed clear enough to read (I did not test them), but I felt than even when teaching by example there needs to be more overview and explanation. I.e., I would prefer an overview of WASM structure and use with examples, rather than just the examples. (I have some (but limited) experience using WASM.)

As for the utility of wasm, note also that Cloudflare workers can run WASM on edge servers [1], and that the Swift community has some support for compiling to wasm [2].

I've never really understood how wasm could do better than java bytecode, but I've been impressed with how much people are using lua and BPF. More generally, in a world of federated programming, we need languages client can submit that providers can run safely, without obviously leaking any secret sauce -- perhaps e.g., for model refinement or augmented lookup.

[1] https://github.com/cloudflare/workers-wasi

Those are great questions! I believe Emscripten will be required for some cases as it provides more features for targeting a Web Browser. If WASI is the only requirement for a Wasm module, then there are three possible solutions:

- Use a library that provides the WASI bindings in a browser environments: there are some OSS projects that provides WASI bindings on top of browser technologies. For example, workers-wasi from Cloudflare [1]. It could be even another Wasm module that provides the implementation for the main one. I know the people from Loophole Labs are experimenting with virtual filesystems (VFS) [2].

- Browsers provides a WASI implementation: server-oriented runtimes like NodeJS are already providing these bindings (under a experimental flag). I shouldn't have stated that as a fact, as browsers may provide it or not. However, I saw in the past the Google Chrome team experimenting with WASI and the browser FileSystem API [3]. So, I think it may happen :)

- [1] https://github.com/cloudflare/workers-wasi

- [2] https://www.youtube.com/watch?v=46jZSXVxYPw

- [3] https://github.com/GoogleChromeLabs/wasi-fs-access

Indeed, some people are doing this:

- WASI once had an official polyfill https://wasi.dev/polyfill/, now apparently succeeded by https://github.com/bjorn3/browser_wasi_shim

- wasmer-js provides a JS polyfill for WASI https://docs.wasmer.io/integrations/js/wasi

- Cloudflare has a WIP polyfill https://github.com/cloudflare/workers-wasi

I'm generally leery of non-temporary polyfills, so I'm not sure that any of these feel like a long-term viable option for me.

I think modern C++ could be perfectly viable as well. Maybe https://github.com/cloudflare/workers-wasi would be a good starting point? I'm not too knowledgeable on the subject. Exciting times though, I think WASM might be the great equalizer.

While there is a WASI implementation for Workers: cloudflare/workers-wasi, I prefer to implement each import manually - especially when there are so few and especially while I am still experimenting. This helps me to keep the full picture of what's going on.

"Rusqlite is an ergonomic wrapper for using SQLite from Rust." - Crates.io

If you're less concerned about the "structure" of your data (e.g., serializing into rust types) and just need tabular data that can be queried (e.g., how much did we bet on X date, who placed a bet on Y team, etc.) I would definitely lean more towards a SQLite database for that kind of work. rusqlite can get you a functional database fairly quickly with a little reading of the documentation (be sure to use the "bundled" feature).

Create a connection per task. WAL is probably a good idea.

Even using SERIALIZED mode, sqlite has multiple APIs which are completely broken if two clients touch the same connection (https://github.com/rusqlite/rusqlite/issues/342#issuecomment...).

Don't bother, just don't share connections between threads and use the regular multi-thread mode (do use that though).

It fails your "ship with a crate" requirement, but when it comes to "csv but too small for a database" it's always worth having a think about SQLite. Of note, the rusqlite crate with the bundled feature will download, compile, and link against sqlite.

You may want to read the Readme of Rusqlite, especially the Optional Features.

As far as I know, the only option for an embedded SQL database is SQLite. The most actively maintained one, for rust, seems to be rusqlite (https://github.com/rusqlite/rusqlite).

Yes, but it's readonly. Also they did not merge loadable extensions support, which I need - https://github.com/rusqlite/rusqlite/pull/910

rusqlite 0.27.0, which looks like it's still the latest version

SQLite is written in C, while workers is based on V8 isolates, so it mainly runs JavaScript. Fortunately, it also supports running WASM through initialising and calling WASM modules via JavaScript. Emscripten can be used to build WASM from C, but I'd rather use it through Rust (using rusqlite), so this is what I focus on right away. Workers can also be written entirely in Rust using worker-rs.