protobuf-go VS quick-protobuf

Protocol Buffers: https://developers.google.com/protocol-buffers

ProtocolBuffers’ OneOf message addresses the case of having a message with many fields where at most one field will be set at the same time.

That's definitely the bigger thing. I think something like Protocol Buffers (Protobuf) is what you're looking for there. Output the data and consume it by something that can handle the analysis.

These protocols prevent an O(N x M) explosion of code that have to solve for many cases. For example, since JSON is an almost ubiquitous format for wire transfer (although other things do exist like protobufs), if I had N data formats that I want to serialize, I only need to write N serializers/deserializers (SerDes). If there was no such narrow waist and there were M alternatives to JSON in wide usage, I would have to write N x M SerDes for wire encoding my data.

So, I thought this at one point, too. But it turns out that methods is a type alias to an unnamed type, so there's no package level privacy issues: https://github.com/protocolbuffers/protobuf-go/blob/v1.26.0/...

[quick-protobuf]: https://github.com/tafia/quick-protobuf

I actually went through all projects listed in [1] because I remember this very quirk. It turns out that there are many such libraries that have two variants of encode/decode functions, where the second variant prepends a varint length. In my brief inspection there do exist a few libraries with only the second variant (e.g. Rust quick-protobuf), which is legitimately problematic [2].

But if the project in question was indeed protobuf.js (see loeg's comments), it clearly distinguishes encode/decode vs. encodeDelimited/decodeDelimited. So I believe the project should not be blamed, and the better question would be why so many people chose to add this exact helper. Well, because Google itself also had the same helper [3]! So at this point protobuf should just standardize this simple framing format (with an explicitly different name though), instead of claiming that protobuf has no obligation to define one.

[1] https://github.com/protocolbuffers/protobuf/blob/main/docs/t...

[2] https://github.com/tafia/quick-protobuf/issues/130

[3] https://protobuf.dev/reference/java/api-docs/com/google/prot...

[4] https://github.com/protocolbuffers/protobuf/blob/main/src/go...