Bond VS FlatBuffers

Microsoft uses Haskell in Bond, a cross-platform framework for working with schematized data.

FlatBuffers, also developed by Google, is a highly optimized serialization library designed for scenarios where zero-copy deserialization is required.

Oh how I enjoy trying to compile and use projects where they use some complex home brew codegen system often written in a different language entirely [1]. Luckily they often use Python as part of some core build step which never breaks compatability in their regex librwry [2].

Yes macros can be a pain and should be limited, but in my experience, a couple hundred lines of macros replaces many thousands of lines code generators with complicated baroque build system integrations (ahem ROS2). The tradeoff is even worse when the language supports templates and compile time operations which can usually replace macros with even less code and are easier to understand. Though at least Go supports codegen properly with support in its official tooling.

1: https://github.com/google/flatbuffers/blob/master/src/idl_ge...

According to the official website:

> Protobuf is intentionally designed to NOT require any parsing at all.

As others have mentioned, this is simply not the case, and the VARINT encoding is a trivial counterexample.

It is this required decoding/parsing that (largely) distinguishes protobuf from Google's flatbuffers:

https://github.com/google/flatbuffers

https://flatbuffers.dev/

Cap'n Proto (developed by Kenton Varda, the former Google engineer who, while at Google, re-wrote/refactored Google's protobuf to later open source it as the library we all know today) is another example of zero-copy (de)serialization.

I don't work at Cloudflare but follow their work and occasionally work on performance sensitive projects.

If I had to guess, they looked at the landscape a bit like I do and regarded Cap'n Proto, flatbuffers, SBE, etc. as being in one category apart from other data formats like Avro, protobuf, and the like.

So once you're committed to record'ish shaped (rather than columnar like Parquet) data that has an upfront parse time of zero (nominally, there could be marshalling if you transmogrify the field values on read), the list gets pretty short.

https://capnproto.org/news/2014-06-17-capnproto-flatbuffers-... goes into some of the trade-offs here.

Cap'n Proto was originally made for https://sandstorm.io/. That work (which Kenton has presumably done at Cloudflare since he's been employed there) eventually turned into Cloudflare workers.

Another consideration: https://github.com/google/flatbuffers/issues/2#issuecomment-...

Much more in the discussion of this particular issue onGitHub: flatbuffers:Reverse engineering #4258

Related but not Rust-specific: FlatBuffers, Cap'n Proto.