gogoprotobuf VS goprotobuf

The Go tooling for gRPC is inexplicably bad, both in terms of ergonomics and in terms of performance.

The GoGoProtobuf [1] project was started to improve both. It would generate nice Go types that followed Go's conventions. And it uses fast binary serialization without needing to resort to reflection.

Unfortunately, the gRPC/Protobuf team(s) at Google is famously resistant to changes, and was unwilling to work with the GoGo. As a result, the GoGo project is now dead. [2]

I've never used Buf, but it looks like it might fix most of the issues with the Go support.

[1] https://github.com/gogo/protobuf

[2] https://x.com/awalterschulze/status/1584553056100057088

That probably won't happen either, because the Google gRPC team is also quite resistant to any kind of change (that isn't driven by internal needs at Google). Projects like GoGo Protobuf had to be developed to fill gaps that the Google team refused to fill or accept PRs for.

the gogo protobuf compiler toolchain is looking for a maintainer https://github.com/gogo/protobuf u/MehdiHK

protobuff : Google's binary data transfer protocol

Looks like Crowdstrike just pushed out publicly a replacement for gogo/proto. It looks like the library solves 2 problems:

Seems like we're pretty middle of the road. I can only guess as to why but it probably has to do with heavy usage of pointers and reflection. Gogo/protobuf solved this performance with code generation, but the go protobuf implementation has essentially eschewed it. I do wonder how the benchmark would look using the new vitess proto library for Go (which has many of the benefits of gogo but with active development and an API built on top of the Google one)

Yeah, it's a pretty thick Makefile and set of shell scripts, but it's seemingly using things like: k8s.io/code-generator/cmd/go-to-protobuf and github.com/gogo/protobuf/protoc-gen-gogo and github.com/gogo/protobuf/protoc-gen-gogofast which from what I can tell... [it looks like this can probably help me a bit in understanding how to make custom types](https://cloud.redhat.com/blog/kubernetes-deep-dive-code-generation-customresources).

repositories: # Definition depends on the 27156597fdf4fb77004434d4409154a230dc9a32 version of https://github.com/googleapis/googleapis # and defines its name as GOOGLE_APIS # It can be referenced in importPaths by $GOOGLE_APIS GOOGLE_APIS: https://github.com/googleapis/googleapis@27156597fdf4fb77004434d4409154a230dc9a32 # Definition depends on the 226206f39bd7276e88ec684ea0028c18ec2c91ae version of https://github.com/gogo/protobuf # and defines its name as GOGO_PROTOBUF # It can be referenced in the importPaths by $GOGO_PROTOBUF GOGO_PROTOBUF: https://github.com/gogo/protobuf@226206f39bd7276e88ec684ea0028c18ec2c91ae

В проекті ми використовуємо офіційну бібліотеку Protobuf github.com/protocolbuffers/protobuf, яка підчас серіалізації використовує рефлексію і будує слайс байтів через append. А потім я дізнався про "Protocol Buffers for Go with Gadgets" github.com/gogo/protobuf, бібліотеку-fork яка генерує додатковий код щоб прибрати рефексію підчас серіалізації і вже записує в слайс байтів по індексу бо так швидше. Коли змінював одну бібліотеку на іншу то важливим вважав що стало працювати швидше і написані раніше тести пройшли успішно. І все б було гаразд але в проекті існувала латка яка через пару тижнів після заміни перезапустила мікросервіс через паніку:

> timestamppb.New(time) is hard to figure out?

No need to be snarky; that API did not exist when I started using protobuf. https://github.com/golang/protobuf/blame/master/ptypes/times... <-- and you can still see the full code from this era on master because of the migration from `github.com/golang/protobuf` to `google.golang.org/protobuf`, which was a whole other exercise in terrible DX.

Now let’s take a look at the source code of the protoc-gen-go plugin:

Wait, you say, in Go you can embed a struct with default method implementations to "inherit" them in your composed struct... sure, except any methods called by those methods are early-bound in the original struct, completely ignoring your wrapper, so the best you can do is "not implemented" rather than actually implement something. It is at least a way to prevent semver-major breakage, which the gRPC generator uses, but that's about as far as it gets you.

The classic answer is protobufs. You can serialize out to binary format.

Seems like go is pretty middle of the road. I can only guess as to why but it probably has to do with heavy usage of pointers and reflection which are much slower than other implementations. Gogo/protobuf (RIP) solved this performance with code generation, but the the official go protobuf implementation has essentially eschewed it. I do wonder how the benchmark would look using the new vitess proto library for Go (which has many of the benefits of gogo but with active development and an API built on top of the Google one)

Maybe I'm missing something, but my read of [golang/protobuf#364](https://github.com/golang/protobuf/issues/364) was that the re-organization in protobuf-go v2 was allow for optimizations like gogoprotobuf to be developed without requiring a complete fork. I totally understand that the authors of gogoprotobuf do not have the time to re-architect their library to use these hooks, but best I can figure this generator does not use these hooks either. Instead it defines additional member functions, and wrappers that look for those specialized functions and fallback to the generic ones if not found.

I am thinking about stuff like the [ProtoMethods](https://pkg.go.dev/google.golang.org/[email protected]/reflec...) API.

I wonder why not? Did the authors of the vtprotobuf extension not want to bite off that much work? Is the new API not sufficient to do what they want (thus failing some of the goals expressed in golang/protobuf#364?

In this case try approach with line by line generation. Very much like what protoc-gen-go does for Go code: https://github.com/golang/protobuf/blob/ae97035608a719c7a1c1c41bed0ae0744bdb0c6f/protoc-gen-go/grpc/grpc.go#L142, need to implement this kind of generator yourself.