postcard VS capnproto-rust

I've been enjoying postcard as a lightweight efficient binary serialization.

That said, postcard is not a self describing format (so everyone needs the same schema, no changes allowed), but I do have a tracking issue open for ways to handle this in the future.

For example, look at the Postcard serializer for inspiration.

Serde should be no_std, I have used it embedded with crates such as postcard. You can see how postcard disables default features of serde here : https://github.com/jamesmunns/postcard/blob/main/Cargo.toml

Another option that may work for some is SerDe on rust. You just write a struct with the fields you want #[derive(Serialize, Deserialize)] above it, and if codegens the functions to deserialize that struct from JSON and serialize it back. Example looks like this https://docs.rs/serde_json/latest/serde_json/#creating-json-by-serializing-data-structures (but you have to use serde-json-core if you don’t have an allocator). Can also easily reserialize to something small like postcard that’s meant for embedded storage.

https://github.com/jamesmunns/postcard

postcard seems like it would be particularly strong for the wasm use case as it produces small messages that are light in memory.

The only hurdle I have is that while the documentation is extensive it is a little confusing in places and mainly focuses on C++ and the C++ RPC system which is a little different to the Rust code. There are Rust examples in the official repo which I will try and leverage here.

capnproto-rust is the official Rust implementation.

Hi, you seem to know a bit about Sanakirja!

It stores 4kb blobs, right? Does Pijul first parses the data (copying it to other allocations), or uses the data as is? I mean, there are some libraries like cap'n'proto[0] and rkyv[1] that can directly use the file contents as an in-memory data structure, I was wondering if Pijul did anything like that.

I mean, is this btree page [2] stored exactly like this on disk, and does Pijul exploits that to avoid further copying data?

(I guess there's a trouble with compression there: to decompress you really need to write in another buffer)

Also, is the I/O done with something that prevent userspace copies like mmap or io_uring, or does it eventually calls read() to copy the data to its own buffer?

I want to build something like Sanakirja, but with those features, so I'm wondering if there's any overlap.

[0] https://github.com/capnproto/capnproto-rust

[1] https://github.com/rkyv/rkyv

[2] https://docs.rs/sanakirja-core/latest/sanakirja_core/btree/p...