verneuil VS s3fs

It'd be process wide. If you want an example can check out the example using a vfs here. There's an explicit passing of vfs there and an implicit usage of it. https://github.com/backtrace-labs/verneuil/blob/main/examples/rusqlite_integration.rs

It doesn't even have to be WAL-based system. Backtrace Labs has a SQLite virtual file system (VFS) called Verneuil that works similarly but works with the rollback journal instead of the WAL.

https://github.com/backtrace-labs/verneuil/ is one way to address the diffing / read replica part of the problem. I believe it's compatible with gossipping: most of the data is in small content-addressed chunks, with small manifests that tell clients what chunks to fetch and how to reassemble them to recreate a sqlite database. There's already client-side caching to persistent storage, and chunks can be fetched on demand.

Sharing replication data P2P, while retaining the simplicity of a single authoritative writer per database, is explicitly part of the project's long-term goals!

- https://github.com/fsspec/s3fs (used by `pandas`, wraps aiobotocore)

I'm using Python's s3fs library to check if a particular file exists in s3 with s3fs.S3FileSystem().exists(path), but I'm getting a Forbidden exception. From the stack trace, I can see it fails when calling s3's head_object method. The documentation for head_object method says: