buf VS googleapis

The Buf CLI is a versatile tool designed for handling Protocol Buffers (Protobuf), a method of serializing structured data. It offers several key features, including managing Protobuf assets through the Buf Schema Registry (BSR), providing a linter to enforce optimal API design and structure, and a breaking change detector to maintain compatibility either in source code or at the wire level. Additionally, the Buf CLI includes a generator that activates plugins based on user-defined templates and a formatter to standardize the formatting of Protobuf files according to industry norms. It also integrates seamlessly with the Buf Schema Registry, supporting comprehensive dependency management.

We'll use the Buf CLI as an alternative to protoc so that we can save our generation configuration as YAML. Buf is compatible with protoc plugins.

Not so much anymore! Take a look at buf.build, it makes the whole thing notoriously easy :)

In my opinion it's a good idea, it's the architecture we use at work, and it works well for us. The main limitation to be aware of is that many PaaS don't support gRPC traffic (because of the proxies used). For example, DigitalOcean App Platform or Heroku if I remember correctly. If the way you want to host your backend is OK with HTTP/2 and gRPC traffic, then it's not a limitation. One way around this limitation is to use the gRPC-Web protocol, or the Connect protocol (https://connect.build/). Unfortunately, Dart's gRPC client does not support the gRPC-Web protocol outside the web platform. So for a mobile application, it's not usable at the moment. (If this PR were accepted, it would solve the issue: https://github.com/grpc/grpc-dart/pull/557.) As for Connect, no client is currently offered by Buf for Dart. Don't hesitate if you want to know more. That said, I'd advise you to use the Connect implementation for Go to implement your backend. Connect will enable your server to speak all three protocols (gRPC, gRPC-Web and Connect), which is very useful in the long term. What's more, the code is cleaner, and you benefit from official support for observability with OpenTelemetry. If you don't know Buf (the creators of Connect),I suggest you visit their website: https://buf.build/. :-) Good luck!

As mentioned in the intro, we are going to use Buf and Connect as our tools. We’ll start by installing the dependencies.

gRPC itself is quite nice, especially with buf which makes generating Go code much easier. The rest of the code was in a bad state. Unmaintained router packages, repository pattern without any actual benefit or a repository pattern.

The second big difference is that we now have auto-generated client and server stubs. For this task, I chose to use buf and the protobuf-ts plugin in order to generate idiomatic Typescript classes and objects. Not only do these classes describe the types we'll use in the server and client, but also includes the actual gRPC implementations used to serialize and send messages back and forth across the wire.

Our team has been using Buf (https://buf.build) recently, and they have a nice solution for schema dependency management.

I've found that https://buf.build/ is easier to use than protoc directly.

Consider checking out https://connect.build from https://buf.build. Supports a simpler protocol than grpc-web. Includes a js/ts client for frontend. Then you don’t necessarily need a rest layer, but could leverage the proxy your building.

Rich Error Model: This model enables servers to return and clients to consume additional error details expressed as one or more protobuf messages. It further specifies a standard set of error message types to cover the most common error (QuotaFailure, PreconditionFailure, BadRequest, etc). When an error occurs, the server returns the appropriate status code along with an optional error message.

They list search in their public api

https://github.com/googleapis/googleapis/blob/288aa7fb71c9b6...

We structure the repo according to proto packages. It's quite similar to how the googleapis repository is structured.

It's unfortunate that the spec doesn't contain custom fields to a sub-object like other RPC specs, like proto Status [1]. They should have had the message go into a field named "message" and not "detail". And have a field like "details" where the opaque type is serialized, which should be named by the "type" field. The problem is that systems with existing error types may have field name conflicts with type, title, status, detail, or instance, so we'd just dump the actual error into a custom "extension member" which by definition, isn't standard.

[1] https://github.com/googleapis/googleapis/blob/1c8a25ab153eef...

Protobuf (https://github.com/googleapis/googleapis)

All the required changes can be viewed in our last demo, the grpc-rest-app implementation. First, we need to update our proto service interface to help the proxy service make our gRPC service methods available at the right URLs and for the correct HTTP operations. To do this, the Google API HTTP library provides annotations we can add to our proto to describe the correct mappings. The buf tool allows us to include the googleapis dependency as a plugin in our buf.yaml file).

I think this only makes sense if the 3rd party is also throwing custom exceptions.

If you want to reduce coupling you should avoid throwing custom exceptions at all. Semantic information can go in the error message and log. The error type should be used to indicate to your program whether an error is recoverable, retriable or some other action needs to be taken. For example google on has 16 canonical error codes for all APIs.

https://github.com/googleapis/googleapis/blob/master/google/...

OpenAPI and possibly developing reusable, versioned client libraries could help, but it's a major undertaking that gRPC makes redundant. I'd be tempted to use grpc-gateway even if I had to implement a REST API. Try looking into buf and monorepo structures for proto management, e.g. something like GoogleCloudPlatform/microservices-demo. For more thorough proto and grpc-gateway definition examples, see googleapis/googleapis.

Firestone does provide global consistency, so the following quote is incorrect:

> In Cloud Firestore, the data on the client are loaded from the database at different points in time. Even if you listen for realtime updates, results from separate queries will not remain in sync. This creates consistency anomalies and bugs in your app.

Here is a link to the protocol documentation that the clients use to support it: https://github.com/googleapis/googleapis/blob/d0b394f188e8c3...

I'd link to the client implementation but it's quite involved.

Well there is Timestamp defined as a well known type which is available to all implementations despite not being a primitive type [1]. Plus one is obviously able to define any other custom types if necessary- eg as seen in [2].

[1] https://developers.google.com/protocol-buffers/docs/referenc...

[2] https://github.com/googleapis/googleapis/blob/master/google/...