go_serialization_benchmarks
MessagePack
go_serialization_benchmarks | MessagePack | |
---|---|---|
8 | 22 | |
1,527 | 1,378 | |
- | 0.3% | |
4.4 | 8.1 | |
12 days ago | 5 days ago | |
Go | Java | |
- | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
go_serialization_benchmarks
-
Rob Pike: Gobs of data (2011)
Someone made a benchmark of serialization libraries in go [1], and I was surprised to see gobs is one of the slowest ones, specially for decoding. I suspect part of the reason is that the API doesn't not allow reusing decoders [2]. From my explorations it seems like both JSON [3], message-pack [4] and CBOR [5] are better alternatives.
By the way, in Go there are a like a million JSON encoders because a lot of things in the std library are not really coded for maximum performance but more for easy of usage, it seems. Perhaps this is the right balance for certain things (ex: the http library, see [6]).
There are also a bunch of libraries that allow you to modify a JSON file "in place", without having to fully deserialize into structs (ex: GJSON/SJSON [7] [8]). This sounds very convenient and more efficient that fully de/serializing if we just need to change the data a little.
--
1: https://github.com/alecthomas/go_serialization_benchmarks
2: https://github.com/golang/go/issues/29766#issuecomment-45492...
--
3: https://github.com/goccy/go-json
4: https://github.com/vmihailenco/msgpack
5: https://github.com/fxamacker/cbor
--
6: https://github.com/valyala/fasthttp#faq
--
7: https://github.com/tidwall/gjson
8: https://github.com/tidwall/sjson
-
Introducing Tempo: low latency, cross-platform, end-to-end typesafe APIs
The bebop definition specifies fixed-width types inside a struct. The format of structs cannot be changed, but there are efficiency gains by omitting all of the indices and header data. It's useless as the root message, but it's small and fast for a benchmark.
-
mus-go - the fastest Golang serializer today
Hey everyone! Let me introduce you to mus-go - the fastest Golang serializer today. If you look at benchmarks (https://github.com/alecthomas/go_serialization_benchmarks), you can see that it could be almost twice as fast as its closest "competitor":
-
What is the fastest way to encode the arbitrary struct into bytes?
This might be of interest: https://github.com/alecthomas/go_serialization_benchmarks
- 80x improvements in caching by moving from JSON to gob
-
gRPC Is Easy to Misconfigure
The protobuf vs msgpack benchmarks are not too bad. Msgpack performs very decently.
https://github.com/alecthomas/go_serialization_benchmarks
-
Bebop encoding in Go
Maybe submit a PR against https://github.com/alecthomas/go_serialization_benchmarks? That covers a ton of serialization formats already, so adding your library would be cool and avoid wheel reinvention.
MessagePack
-
What is the fastest way to encode the arbitrary struct into bytes?
so appreciate such a detailed reply, thanks. btw, why did you choose tinylib/msgp from 4 available go-impls?
-
Using Arduino as input to Rust project (help needed)
If you find you're running the serial connection at maximum speed and it's still not fast enough, try switching to a more compact binary encoding that has both Serde and Arduino implementations, like MsgPack... though I don't remember enough about its format off the top of my head to tell you the easiest way to put an unambiguous header on each packet/message to make the protocol self-synchronizing.
-
Java Serialization with Protocol Buffers
The information can be stored in a database or as files, serialized in a standard format and with a schema agreed with your Data Engineering team. Depending on your information and requirements, it can be as simple as CSV, XML or JSON, or Big Data formats such as Parquet, Avro, ORC, Arrow, or message serialization formats like Protocol Buffers, FlatBuffers, MessagePack, Thrift, or Cap'n Proto.
-
Multiplayer Networking Solutions
MessagePack Similar to JSONs, just more compact, although not as much as the ones above. Still, it's usefull to retain some readability in your messages.
-
Sketch crashes with "Soft WDT reset" randomly (ArduinoJSON and HTTPClient)
I'll try that msgpack.org website.
- Unknown encryption method ?
-
GitHub - realtimetech-solution/opack: Fast object or data serialize and deserialize library
First of all, you're comparing this to GSON and Kryo, how does it compare to Msgpack, fast-serialization, but also Elsa and I'm sure, many others? Are there any limitations and/or trade-offs?
-
Optimal dispatcher for json messages ?
Upvote for msgpack, one of the great undervalued message protocols available.
-
Rust is just as fast as C/C++
I have two suggestions Capnproto, MessagePack (those are only the two examples that came to mind first, i bet there are even one or two especially developed for rust). Both of these are better than json in nearly every way.
-
msgspec - a fast & friendly JSON/MessagePack library
Encode messages as JSON or MessagePack.
What are some alternatives?
bebop - bebop wire format in Go
FlatBuffers - FlatBuffers: Memory Efficient Serialization Library
encoding - Go package containing implementations of efficient encoding, decoding, and validation APIs.
Kryo - Java binary serialization and cloning: fast, efficient, automatic
bebop - 🎷No ceremony, just code. Blazing fast, typesafe binary serialization.
Cap'n Proto - Cap'n Proto serialization/RPC system - core tools and C++ library
msgpack - MessagePack is an extremely efficient object serialization library. It's like JSON, but very fast and small.
Protobuf - Protocol Buffers - Google's data interchange format
grpc-web - gRPC for Web Clients
protostuff - Java serialization library, proto compiler, code generator
msgp - A Go code generator for MessagePack / msgpack.org[Go]
ZLib - A massively spiffy yet delicately unobtrusive compression library.