A new ProtoBuf generator for Go

This page summarizes the projects mentioned and recommended in the original post on news.ycombinator.com

Our great sponsors
  • SonarQube - Static code analysis for 29 languages.
  • Scout APM - Less time debugging, more time building
  • SaaSHub - Software Alternatives and Reviews
  • gapid

    Graphics API Debugger

    > Arenas are, however, unfeasible to implement in Go because it is a garbage collected language.

    If you are willing to use cgo, google already implemented one for gapid.

    https://github.com/google/gapid/tree/master/core/memory/aren...

  • bumpalo

    A fast bump allocation arena for Rust

    Rust has an arena allocator too[1], but it is implemented with 165(!!!) usages of unsafe. :)

    [1] https://github.com/fitzgen/bumpalo

  • SonarQube

    Static code analysis for 29 languages.. Your projects are multi-language. So is SonarQube analysis. Find Bugs, Vulnerabilities, Security Hotspots, and Code Smells so you can release quality code every time. Get started analyzing your projects today for free.

  • generational-arena

    A safe arena allocator that allows deletion without suffering from the ABA problem by using generational indices.

    This is far from the only arena allocator written in Rust.

    From the same author, a zero-unsafe arena allocator: https://github.com/fitzgen/generational-arena

    There are many, many arena implementations available with varying characteristics. It's disingenuous to act like Rust requires the author of an arena library to write "unsafe" everywhere.

  • goprotobuf

    Go support for Google's protocol buffers (by golang)

    Maybe I'm missing something, but my read of [golang/protobuf#364](https://github.com/golang/protobuf/issues/364) was that the re-organization in protobuf-go v2 was allow for optimizations like gogoprotobuf to be developed without requiring a complete fork. I totally understand that the authors of gogoprotobuf do not have the time to re-architect their library to use these hooks, but best I can figure this generator does not use these hooks either. Instead it defines additional member functions, and wrappers that look for those specialized functions and fallback to the generic ones if not found.

    I am thinking about stuff like the [ProtoMethods](https://pkg.go.dev/google.golang.org/[email protected]/reflec...) API.

    I wonder why not? Did the authors of the vtprotobuf extension not want to bite off that much work? Is the new API not sufficient to do what they want (thus failing some of the goals expressed in golang/protobuf#364?

  • protobuf-go

    Go support for Google's protocol buffers

    So, I thought this at one point, too. But it turns out that methods is a type alias to an unnamed type, so there's no package level privacy issues: https://github.com/protocolbuffers/protobuf-go/blob/v1.26.0/...

  • Protobuf

    Protocol Buffers - Google's data interchange format

    Funny timing, I've just written most of a TypeScript generator for protobufs. I learned about some fun corners of protobufs I didn't expect trying to pass the protouf conformance tests [1] (which this one passes, that's no mean feat!).

    - If you write the same message multiple times, protobuf implementations should merge fields with a last write wins policy (repeated fields are concatenated). This includes messages in oneofs.

    - For a boolean array, you're better off using a packed, repeated int64 (if wire size matters a lot). Protobuf bools use varint encoding meaning you need at least 2 bytes for every boolean, 1+ for the tag and type and 1 byte for the 0 or 1 value. With a repeated int64, you'd encode the tag and length in 2 varints, and then you get 64 bools per 8 bytes.

    - Fun trivia: Varints take up a max of 10 bytes but could be implemented in 9 bytes. You get 7 bits per varint byte, so 9 bytes gets you 63 bits. Then you could use the most significant bit of the last byte to indicate if the last bit is 0 or 1. Learned by reading the Go varint implementation [2].

    - Messages can be recursive. This is easy if you represent messages as pointers since you can use nil. It's a fair bit harder if you want to always use a value object for each nested message since you need to break cycles by marking fields as `T | undefined` to avoid blowing the stack. Figuring out the minimal number of fields to break cycles is an NP hard problem called the minimum feedback arc set[3].

    - If you're writing a protobuf implementation, the conformance tests are a really nice way to check that you've done a good job. Be wary of implementations that don't implement the conformance tests.

    [1]: https://github.com/protocolbuffers/protobuf/tree/master/conf...

    [2]: https://github.com/golang/go/blob/master/src/encoding/binary...

    [3]: https://en.wikipedia.org/wiki/Feedback_arc_set#Minimum_feedb...

  • go

    The Go programming language

    Funny timing, I've just written most of a TypeScript generator for protobufs. I learned about some fun corners of protobufs I didn't expect trying to pass the protouf conformance tests [1] (which this one passes, that's no mean feat!).

    - If you write the same message multiple times, protobuf implementations should merge fields with a last write wins policy (repeated fields are concatenated). This includes messages in oneofs.

    - For a boolean array, you're better off using a packed, repeated int64 (if wire size matters a lot). Protobuf bools use varint encoding meaning you need at least 2 bytes for every boolean, 1+ for the tag and type and 1 byte for the 0 or 1 value. With a repeated int64, you'd encode the tag and length in 2 varints, and then you get 64 bools per 8 bytes.

    - Fun trivia: Varints take up a max of 10 bytes but could be implemented in 9 bytes. You get 7 bits per varint byte, so 9 bytes gets you 63 bits. Then you could use the most significant bit of the last byte to indicate if the last bit is 0 or 1. Learned by reading the Go varint implementation [2].

    - Messages can be recursive. This is easy if you represent messages as pointers since you can use nil. It's a fair bit harder if you want to always use a value object for each nested message since you need to break cycles by marking fields as `T | undefined` to avoid blowing the stack. Figuring out the minimal number of fields to break cycles is an NP hard problem called the minimum feedback arc set[3].

    - If you're writing a protobuf implementation, the conformance tests are a really nice way to check that you've done a good job. Be wary of implementations that don't implement the conformance tests.

    [1]: https://github.com/protocolbuffers/protobuf/tree/master/conf...

    [2]: https://github.com/golang/go/blob/master/src/encoding/binary...

    [3]: https://en.wikipedia.org/wiki/Feedback_arc_set#Minimum_feedb...

  • Scout APM

    Less time debugging, more time building. Scout APM allows you to find and fix performance issues with no hassle. Now with error monitoring and external services monitoring, Scout is a developer's best friend when it comes to application development.

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts