cbor
goprotobuf
Our great sponsors
cbor | goprotobuf | |
---|---|---|
4 | 13 | |
658 | 9,534 | |
- | 0.7% | |
8.6 | 2.8 | |
2 days ago | about 1 month ago | |
Go | Go | |
MIT License | BSD 3-clause "New" or "Revised" License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
cbor
-
Rob Pike: Gobs of data (2011)
Someone made a benchmark of serialization libraries in go [1], and I was surprised to see gobs is one of the slowest ones, specially for decoding. I suspect part of the reason is that the API doesn't not allow reusing decoders [2]. From my explorations it seems like both JSON [3], message-pack [4] and CBOR [5] are better alternatives.
By the way, in Go there are a like a million JSON encoders because a lot of things in the std library are not really coded for maximum performance but more for easy of usage, it seems. Perhaps this is the right balance for certain things (ex: the http library, see [6]).
There are also a bunch of libraries that allow you to modify a JSON file "in place", without having to fully deserialize into structs (ex: GJSON/SJSON [7] [8]). This sounds very convenient and more efficient that fully de/serializing if we just need to change the data a little.
--
1: https://github.com/alecthomas/go_serialization_benchmarks
2: https://github.com/golang/go/issues/29766#issuecomment-45492...
--
3: https://github.com/goccy/go-json
4: https://github.com/vmihailenco/msgpack
5: https://github.com/fxamacker/cbor
--
6: https://github.com/valyala/fasthttp#faq
--
-
What is the fastest way to encode the arbitrary struct into bytes?
Ha, no suggestions for cbor https://github.com/fxamacker/cbor
- How can we umarshal a Big JSON effectively?
-
80x improvements in caching by moving from JSON to gob
If you need to support non-Go code I suggest you explore CBOR. There are a few implementations in Go but his is my favorite.
goprotobuf
-
Protoc Plugins with Go
Now let’s take a look at the source code of the protoc-gen-go plugin:
- How Turborepo is porting from Go to Rust
-
The Tragic Death of Inheritance
Wait, you say, in Go you can embed a struct with default method implementations to "inherit" them in your composed struct... sure, except any methods called by those methods are early-bound in the original struct, completely ignoring your wrapper, so the best you can do is "not implemented" rather than actually implement something. It is at least a way to prevent semver-major breakage, which the gRPC generator uses, but that's about as far as it gets you.
- Protobuf - Go support for Google's protocol buffers
-
Passing large amounts of data between processes via a file?
The classic answer is protobufs. You can serialize out to binary format.
-
2022-01-11 gRPC benchmark results
Seems like go is pretty middle of the road. I can only guess as to why but it probably has to do with heavy usage of pointers and reflection which are much slower than other implementations. Gogo/protobuf (RIP) solved this performance with code generation, but the the official go protobuf implementation has essentially eschewed it. I do wonder how the benchmark would look using the new vitess proto library for Go (which has many of the benefits of gogo but with active development and an API built on top of the Google one)
- A complete yet beginner friendly guide on how to secure Linux
-
A new ProtoBuf generator for Go
Maybe I'm missing something, but my read of [golang/protobuf#364](https://github.com/golang/protobuf/issues/364) was that the re-organization in protobuf-go v2 was allow for optimizations like gogoprotobuf to be developed without requiring a complete fork. I totally understand that the authors of gogoprotobuf do not have the time to re-architect their library to use these hooks, but best I can figure this generator does not use these hooks either. Instead it defines additional member functions, and wrappers that look for those specialized functions and fallback to the generic ones if not found.
I am thinking about stuff like the [ProtoMethods](https://pkg.go.dev/google.golang.org/[email protected]/reflec...) API.
I wonder why not? Did the authors of the vtprotobuf extension not want to bite off that much work? Is the new API not sufficient to do what they want (thus failing some of the goals expressed in golang/protobuf#364?
-
How to Auto Generate JavaScript code using GO
In this case try approach with line by line generation. Very much like what protoc-gen-go does for Go code: https://github.com/golang/protobuf/blob/ae97035608a719c7a1c1c41bed0ae0744bdb0c6f/protoc-gen-go/grpc/grpc.go#L142, need to implement this kind of generator yourself.
-
Writing a code generator in Go
Something like this: https://github.com/golang/protobuf/blob/master/internal/gengogrpc/grpc.go
What are some alternatives?
asn1
colfer - binary serialization format
gogoprotobuf - [Deprecated] Protocol Buffers for Go with Gadgets
go-capnproto - Cap'n Proto library and parser for go. This is go-capnproto-1.0, and does not have rpc. See https://github.com/zombiezen/go-capnproto2 for 2.0 which has rpc and capabilities.
jsoniter - A high-performance 100% compatible drop-in replacement of "encoding/json"
go-codec - idiomatic codec and rpc lib for msgpack, cbor, json, etc. msgpack.org[Go]
mapstructure - Go library for decoding generic map values into native Go structures and vice versa.
fwencoder - Fixed width file parser (encoder/decoder) in GO (golang)