serde-repr VS bincode

The other issue is that it looks like serde_repr will not generate correct code for #[repr(C)] enums as it appears to interpret the bit inside #[repr(...)] as an integer type: https://github.com/dtolnay/serde-repr/blob/master/src/lib.rs#L60

Now we have 500 small datasets and an index that maps centroid points to the relevant dataset. Next, to enable real-time performance, we want to precompile search graphs so that we don't need to initialize and construct them at runtime, and can use as little CPU time as possible. A really fast nearest-neighbor algorithm is Hierarchical Navigable Small Worlds (HNSW), and it has a pure Rust implementation, which we're using to write our edge app. So we wrote a small standalone Rust app to construct the HNSW graph structs for each dataset, and then used bincode to export the memory of the instantiated struct into a binary blob.

Ermm... actually I meant something like this: playground, but then I realized it's basically (de)serialization, and I just found that we already have a crate for that: bincode.

One, figure out the bincode format (documented here: https://github.com/bincode-org/bincode/blob/trunk/docs/spec.md) and write your own parser. Maybe a one-off that specifically only handles this one data structure would be fairly straightforward.

Step 1: The Deserialize type requests data from the Deserializer with one of the deserialize_type methods. This gives it an opportunity to provide certain metadata about the type: structs provide a list of fields, enums provide a list of variants, tuples provide a length, etc. Some data formats (notably bincode) require this metadata to drive deserializing, as the wire format is not self-describing. Crucially, the Deserialize type also provides a visitor that is capable of receiving the requested data from the Deserializer.

Alternatively, give Bincode a try.

Like separate instructions? I was thinking if a instruction have unknown length I make sure I have some kind of header field that tells the data length of the instruction so receiver knows when next instruction starts. And I was planning on using Bincode with serde to serialize and dezerialize like structs and stuff.

Isn't this essentially bincode?

This is similar in practice to using abi_stable, and end-users will still receive compiled files, but your plugins will be sandboxed and a single build will work on all platforms. The downside is that it's a bit more work because WebAssembly's support for passing complex data types between the host and the WebAssembly code is in the preliminary stages, so you need to do something like using Serde to encode your data into something like Bincode or MessagePack (or JSON and friends) to hand it off between the host and the plugin.