goprotobuf VS buf

The classic answer is protobufs. You can serialize out to binary format.

Seems like go is pretty middle of the road. I can only guess as to why but it probably has to do with heavy usage of pointers and reflection which are much slower than other implementations. Gogo/protobuf (RIP) solved this performance with code generation, but the the official go protobuf implementation has essentially eschewed it. I do wonder how the benchmark would look using the new vitess proto library for Go (which has many of the benefits of gogo but with active development and an API built on top of the Google one)

go get github.com/golang/protobuf/protoc-gen-go

Maybe I'm missing something, but my read of [golang/protobuf#364](https://github.com/golang/protobuf/issues/364) was that the re-organization in protobuf-go v2 was allow for optimizations like gogoprotobuf to be developed without requiring a complete fork. I totally understand that the authors of gogoprotobuf do not have the time to re-architect their library to use these hooks, but best I can figure this generator does not use these hooks either. Instead it defines additional member functions, and wrappers that look for those specialized functions and fallback to the generic ones if not found.

I am thinking about stuff like the [ProtoMethods](https://pkg.go.dev/google.golang.org/[email protected]/reflec...) API.

I wonder why not? Did the authors of the vtprotobuf extension not want to bite off that much work? Is the new API not sufficient to do what they want (thus failing some of the goals expressed in golang/protobuf#364?

In this case try approach with line by line generation. Very much like what protoc-gen-go does for Go code: https://github.com/golang/protobuf/blob/ae97035608a719c7a1c1c41bed0ae0744bdb0c6f/protoc-gen-go/grpc/grpc.go#L142, need to implement this kind of generator yourself.

Something like this: https://github.com/golang/protobuf/blob/master/internal/gengogrpc/grpc.go

The difference to the client-server solution is subtile when it comes to the details in implementation. protoc-gen-go and other plugins in the Protocol Buffers ecosystem use this approach.

Yeah I don't like that as well for professional solutions I like using https://buf.build/

Big fan of gRPC-gateway. Not a big fan of protocol buffers however. That being said, https://buf.build/ makes them manageable and was built by the same guys doing gRPC-gateway

You could also take a look at https://buf.build These guys are working on a protobuf schema registry. I haven’t tried it yet but looks promising.

To maintain backward/forward compatibility in Protobufs, every change to the schema must be thoroughly code-reviewed and tested for compliance. Humans are prone to making errors. Therefore, several tools have emerged to help with the process, most notably Buf.

Everyone I've worked with save a few that experienced protobufs with extentions immediately finds the benefit once they use them.

However I have yet to convince anyone at an org that isn't already using them that they will change things for the better.

Schema once, code everywhere. It is really nice.

Have you seen Buf? https://buf.build/

There is a company Buf which provides the Buf Schema Registry.

No need for protoc, just use buf.

That was a fun read! Have you checked out https://buf.build/ by any chance? It simplifies a lot of the weird/annoying protoc stuff and enforces a schema that other tools can utilize. It also has pattern matching and all that + supports the usual proto plugins.

Hello! I'm the head of developer experience at [Buf](https://buf.build) and I wanted to introduce our schema registry here. I know that this forum tends to revolve around "old-fashioned" REST approaches and GraphQL but I thought I'd inject something a little different. If you've found Protocol Buffers intriguing in the past but weren't satisfied with the tools available, it might be a good time to dip back into the ecosystem. More than happy to answer any questions and address feedback.

In order to get Protobuf generation working, we'll use Buf, a tool specifically built for building protocol buffers. Start by making the following buf.gen.yaml: