goprotobuf VS buf

Now let’s take a look at the source code of the protoc-gen-go plugin:

Wait, you say, in Go you can embed a struct with default method implementations to "inherit" them in your composed struct... sure, except any methods called by those methods are early-bound in the original struct, completely ignoring your wrapper, so the best you can do is "not implemented" rather than actually implement something. It is at least a way to prevent semver-major breakage, which the gRPC generator uses, but that's about as far as it gets you.

The classic answer is protobufs. You can serialize out to binary format.

Seems like go is pretty middle of the road. I can only guess as to why but it probably has to do with heavy usage of pointers and reflection which are much slower than other implementations. Gogo/protobuf (RIP) solved this performance with code generation, but the the official go protobuf implementation has essentially eschewed it. I do wonder how the benchmark would look using the new vitess proto library for Go (which has many of the benefits of gogo but with active development and an API built on top of the Google one)

Maybe I'm missing something, but my read of [golang/protobuf#364](https://github.com/golang/protobuf/issues/364) was that the re-organization in protobuf-go v2 was allow for optimizations like gogoprotobuf to be developed without requiring a complete fork. I totally understand that the authors of gogoprotobuf do not have the time to re-architect their library to use these hooks, but best I can figure this generator does not use these hooks either. Instead it defines additional member functions, and wrappers that look for those specialized functions and fallback to the generic ones if not found.

I am thinking about stuff like the [ProtoMethods](https://pkg.go.dev/google.golang.org/[email protected]/reflec...) API.

I wonder why not? Did the authors of the vtprotobuf extension not want to bite off that much work? Is the new API not sufficient to do what they want (thus failing some of the goals expressed in golang/protobuf#364?

In this case try approach with line by line generation. Very much like what protoc-gen-go does for Go code: https://github.com/golang/protobuf/blob/ae97035608a719c7a1c1c41bed0ae0744bdb0c6f/protoc-gen-go/grpc/grpc.go#L142, need to implement this kind of generator yourself.

Something like this: https://github.com/golang/protobuf/blob/master/internal/gengogrpc/grpc.go

The Buf CLI is a versatile tool designed for handling Protocol Buffers (Protobuf), a method of serializing structured data. It offers several key features, including managing Protobuf assets through the Buf Schema Registry (BSR), providing a linter to enforce optimal API design and structure, and a breaking change detector to maintain compatibility either in source code or at the wire level. Additionally, the Buf CLI includes a generator that activates plugins based on user-defined templates and a formatter to standardize the formatting of Protobuf files according to industry norms. It also integrates seamlessly with the Buf Schema Registry, supporting comprehensive dependency management.

We'll use the Buf CLI as an alternative to protoc so that we can save our generation configuration as YAML. Buf is compatible with protoc plugins.

Not so much anymore! Take a look at buf.build, it makes the whole thing notoriously easy :)

In my opinion it's a good idea, it's the architecture we use at work, and it works well for us. The main limitation to be aware of is that many PaaS don't support gRPC traffic (because of the proxies used). For example, DigitalOcean App Platform or Heroku if I remember correctly. If the way you want to host your backend is OK with HTTP/2 and gRPC traffic, then it's not a limitation. One way around this limitation is to use the gRPC-Web protocol, or the Connect protocol (https://connect.build/). Unfortunately, Dart's gRPC client does not support the gRPC-Web protocol outside the web platform. So for a mobile application, it's not usable at the moment. (If this PR were accepted, it would solve the issue: https://github.com/grpc/grpc-dart/pull/557.) As for Connect, no client is currently offered by Buf for Dart. Don't hesitate if you want to know more. That said, I'd advise you to use the Connect implementation for Go to implement your backend. Connect will enable your server to speak all three protocols (gRPC, gRPC-Web and Connect), which is very useful in the long term. What's more, the code is cleaner, and you benefit from official support for observability with OpenTelemetry. If you don't know Buf (the creators of Connect),I suggest you visit their website: https://buf.build/. :-) Good luck!

As mentioned in the intro, we are going to use Buf and Connect as our tools. We’ll start by installing the dependencies.

gRPC itself is quite nice, especially with buf which makes generating Go code much easier. The rest of the code was in a bad state. Unmaintained router packages, repository pattern without any actual benefit or a repository pattern.

The second big difference is that we now have auto-generated client and server stubs. For this task, I chose to use buf and the protobuf-ts plugin in order to generate idiomatic Typescript classes and objects. Not only do these classes describe the types we'll use in the server and client, but also includes the actual gRPC implementations used to serialize and send messages back and forth across the wire.

Our team has been using Buf (https://buf.build) recently, and they have a nice solution for schema dependency management.

I've found that https://buf.build/ is easier to use than protoc directly.

Consider checking out https://connect.build from https://buf.build. Supports a simpler protocol than grpc-web. Includes a js/ts client for frontend. Then you don’t necessarily need a rest layer, but could leverage the proxy your building.