kowl VS gogen-avro

View on GitHub

Maybe give kowl a try: https://github.com/cloudhut/kowl

More concrete, take a look at Kowl how it uses Franz Go to talk to the Admin API: https://github.com/cloudhut/kowl/blob/master/backend/pkg/kafka/list_consumer_group_offsets.go#L15-L22

This seems pretty confused. The "compiled vs dynamic" distinction is a property of the implementation, not of the protocol.

For example, you can certainly compile Avro into Go source files [0]. You can even compile Avro loaded schemas _during runtime_ into Python bytecode, since Python is interpreted [1]. This even works if you have the _wrong schema document_ for the message (you'll just get the subset of fields which are accurately described), because of Avro's schema compatibility rules.

Likewise, you can deserialize arbitrary protobuf messages during runtime without a compilation step, if you have a description for the message schema. The Python protobuf library has had a "ParseMessage" API forever, and protoreflect [2] exists for Go. (In case it's not obvious, I mostly work in Python and Go but I am completely certain analogues exist in other major languages).

There is a very big and important difference between a protocol and the implementation of a protocol. I think this README's author is not clear on that difference, which shows up in other claims ("Deserialization is incrimental", for example) too.

---

[0] https://github.com/actgardner/gogen-avro

[1] https://github.com/spenczar/avroc

[2] https://pkg.go.dev/google.golang.org/protobuf/reflect/protor...

at my company, we ended up writing a client on top of the Confluent client (which hooks into the librdkafka C library) and uses the gogen-avro library to provide type safe codegen for Avro schemas and is used for the serde process within the client. the API ended up looking basically like the standard Java API. as for feature parity, librdkafka seems to provide most of what we've needed from a Kafka producer/consumer config side of things while the SR implementation is limited in scope anyways and can easily be included in a basic library.