SBE VS Bitsery

I was about to suggest Chronicle, but it looks like they have gone closed-source. The older version is still interesting to look through though. Aeron / Disruptor / SBE are good projects for inspiration as well.

Could you evaluate how it compares with SBE?

The Simple Binary Encoding (SBE) project now includes support for generating Rust code. Generated code produced does not use unsafe and has no dependencies on any other crates.

SBE

Consider a valid protobuf message with such a field. If you can locate the field value bytes, you can write a new value to the same location without breaking the message. It's obviously possible to the same with the varint type too, as long as you don't change the number of bytes - not so practical, but useful for enum field which has a limited set of useful values (usually less than 128).

Pregenerating protobuf messages you want to send and then modifying the bytes in-place before sending is going to give you a nice performance boost over "normal" protobuf serialization. It can be useful if you need to be protobuf compatible, but it's obviously better to use something like SBE - https://github.com/real-logic/simple-binary-encoding

Thanks for sharing https://github.com/fraillt/bitsery it seems more powerful but require quite a boilerplate code for even very simple composite stuff. With ByteStream you can totally forget about deep you are in the data-structure hierarchy.

Bitsery is the closest I've seen to it, but you have to use extensions (which i can't yet figure out the API for) to make it achieve byte for byte compatbility

If size and performance matter, then take a look at bitsery. * it might be 9x+ faster than cereal and 18x faster than protobuf. * size-wise, you might save 20-30% by default * on top of that you additional might opt-in into: * bit-level serialization control (e.g. if your values are in the range 1000-2000 it will take you 10bits, or use VLE) * backward-forward compatibility support * pointer support, including raw pointers with the ability to provide custom allocator. * and powerful extensions system, which allows you to further customize things in any way you want it :)

About on par with nlohmann is a binary serialization lib called Bitsery. It's easy to use, and will be a lot more performant than json. The downside is that the serialized data won't be human readable.