jvm-serializers VS honeycomb-opentelemetry-go

Compared with protobuf, fury is 3.2x faster. When comparing with avro, fury is 5.3x faster. Compared with flatbuffers, fury is 4.8x faster. See https://github.com/eishay/jvm-serializers/wiki for detailed benchmark data

First, there's benchmarks here if you haven't seen it: jvm-serializers. Not terribly scientific, but it's something. To make any decision, you really need to benchmark your own object graph and it's important to configure the serializer for your particular usage. Still, it is sort of useful for comparing frameworks. It would be interesting to see how Loial performs there. Ping me if you add it.

It depends. Some binary encodings such as flatbuffer are actually slower than some JSON libraries. There's a wide range of performance even in the JSON libraries themselves. Generally the faster JSON libraries are the ones that work on a predefined schema and so are able to generate code specifically for that JSON.

> I'm surprised this is up for debate.

I looked into logging in protobuf when I was seeing if there was a better binary encoding for ring-buffer logging, along the same lines as nanolog:

https://tersesystems.com/blog/2020/11/26/queryable-logging-w...

What I found was that it's typically not the binary encoding vs string encoding that makes a difference. The biggest factors are "is there a predefined schema", "is there a precompiler that will generate code for this schema", and "what is the complexity of the output format". With that in mind, if you are dealing with chaotic semi-structured data, JSON is pretty good, and actually faster than some binary encodings:

https://github.com/eishay/jvm-serializers/wiki/Newer-Results...

You could use any of the JVM serialisers which should still work.

I see! Yeah, this is one where where otel-go is a lot harder to use, but it's something the SIG is looking at. A coworker of mine is helping drive a design that's sort of an "easy button" to configure all the things with the least-surprising defaults[0] and we're seeing how people like it in our SDK distribution that uses it[1]. I hope that sometime soon we'll have the design polished-up enough to get merged in. Like most OSS projects, it'll take some time but I'm confident we can get it done.

The main challenge is that there's a large variety of use cases to fulfill (e.g., someone wants custom context propagation, a custom span processor, and export over HTTP+json but not HTTP+protobuf) and today the answer to that is that you have to pull in all the libraries for all the things you need. It's a lot more energy you need to expend to get started with all of this than it needs to be.

As for logging support in the Go SDK, it's frozen mostly just due to lack of bandwidth and a need to finish what's already been started. Metrics have proven to be much more difficult and time-consuming to implement correctly across all languages, with Go being impacted harder than other languages (e.g., Python and .NET). I think you can expect logging integrations in the near-ish future though.

This is great feedback. I'll pass it on folks who haven't seen it. Thank you! And please feel free to file issues about all the things that rub you the wrong way

[0]: https://github.com/open-telemetry/opentelemetry-go-contrib/p...

[1]: https://github.com/honeycombio/honeycomb-opentelemetry-go