Go Serialization

Open-source Go projects categorized as Serialization
Edit details
Topics: Go Golang go-capnproto JSON Encoding
Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com
Sponsored

Top 23 Go Serialization Projects

  • jsoniter

    A high-performance 100% compatible drop-in replacement of "encoding/json" (by json-iterator)

    • Project mention: Handling high-traffic HTTP requests with JSON payloads | /r/golang | 2023-12-07

    Since most of the time would be spent decoding json, you could try to cut this time using https://github.com/bytedance/sonic or https://github.com/json-iterator/go, both are drop-in replacements for the stdlib, sonic is faster.

  • goprotobuf

    Go support for Google's protocol buffers (by golang)

    • Project mention: Protoc Plugins with Go | dev.to | 2023-08-19

    Now let’s take a look at the source code of the protoc-gen-go plugin:

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo

  • mapstructure

    Go library for decoding generic map values into native Go structures and vice versa.

    • Project mention: How do I marshal a JSON array into a map? | /r/golang | 2023-12-07

  • gogoprotobuf

    [Deprecated] Protocol Buffers for Go with Gadgets (by gogo)

  • msgpack

    msgpack.org[Go] MessagePack encoding for Golang (by vmihailenco)

    • Project mention: Rob Pike: Gobs of data (2011) | news.ycombinator.com | 2023-12-04

    Someone made a benchmark of serialization libraries in go [1], and I was surprised to see gobs is one of the slowest ones, specially for decoding. I suspect part of the reason is that the API doesn't not allow reusing decoders [2]. From my explorations it seems like both JSON [3], message-pack [4] and CBOR [5] are better alternatives.

    By the way, in Go there are a like a million JSON encoders because a lot of things in the std library are not really coded for maximum performance but more for easy of usage, it seems. Perhaps this is the right balance for certain things (ex: the http library, see [6]).

    There are also a bunch of libraries that allow you to modify a JSON file "in place", without having to fully deserialize into structs (ex: GJSON/SJSON [7] [8]). This sounds very convenient and more efficient that fully de/serializing if we just need to change the data a little.

    --

    1: https://github.com/alecthomas/go_serialization_benchmarks

    2: https://github.com/golang/go/issues/29766#issuecomment-45492...

    --

    3: https://github.com/goccy/go-json

    4: https://github.com/vmihailenco/msgpack

    5: https://github.com/fxamacker/cbor

    --

    6: https://github.com/valyala/fasthttp#faq

    --

    7: https://github.com/tidwall/gjson

    8: https://github.com/tidwall/sjson

  • go-codec

    idiomatic codec and rpc lib for msgpack, cbor, json, etc. msgpack.org[Go] (by ugorji)

  • hprose

    Hprose is a cross-language RPC. This project is Hprose for Golang.

  • WorkOS

    The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

    WorkOS logo

  • go-capnp

    Cap'n Proto library and code generator for Go

    • Project mention: Sandstorm: Open-source platform for self-hosting web app | news.ycombinator.com | 2023-06-04

    The Go implementation of capnp is alive and well, too. I highly recommend it: https://github.com/capnproto/go-capnp

  • csvutil

    csvutil provides fast and idiomatic mapping between CSV and Go (golang) values.

  • cbor

    CBOR codec (RFC 8949) with CBOR tags, Go struct tags (toarray, keyasint, omitempty), float64/32/16, big.Int, and fuzz tested billions of execs.

    • Project mention: Rob Pike: Gobs of data (2011) | news.ycombinator.com | 2023-12-04

    Someone made a benchmark of serialization libraries in go [1], and I was surprised to see gobs is one of the slowest ones, specially for decoding. I suspect part of the reason is that the API doesn't not allow reusing decoders [2]. From my explorations it seems like both JSON [3], message-pack [4] and CBOR [5] are better alternatives.

    By the way, in Go there are a like a million JSON encoders because a lot of things in the std library are not really coded for maximum performance but more for easy of usage, it seems. Perhaps this is the right balance for certain things (ex: the http library, see [6]).

    There are also a bunch of libraries that allow you to modify a JSON file "in place", without having to fully deserialize into structs (ex: GJSON/SJSON [7] [8]). This sounds very convenient and more efficient that fully de/serializing if we just need to change the data a little.

    --

    1: https://github.com/alecthomas/go_serialization_benchmarks

    2: https://github.com/golang/go/issues/29766#issuecomment-45492...

    --

    3: https://github.com/goccy/go-json

    4: https://github.com/vmihailenco/msgpack

    5: https://github.com/fxamacker/cbor

    --

    6: https://github.com/valyala/fasthttp#faq

    --

    7: https://github.com/tidwall/gjson

    8: https://github.com/tidwall/sjson

  • karmem

    Karmem is a fast binary serialization format, faster than Google Flatbuffers and optimized for TinyGo and WASM.

  • go-capnproto

    Cap'n Proto library and parser for go. This is go-capnproto-1.0, and does not have rpc. See https://github.com/zombiezen/go-capnproto2 for 2.0 which has rpc and capabilities.

  • watson

    WATSON: Wasted but Amazing Turing-incomplete Stack-based Object Notation (by genkami)

  • php_session_decoder

    PHP session encoder/decoder written in Go

  • structomap

    Easily and dynamically generate maps from Go static structures

  • binstruct

    Golang binary decoder for mapping data into the structure

  • bebop

    bebop wire format in Go (by 200sc)

    • Project mention: Introducing Tempo: low latency, cross-platform, end-to-end typesafe APIs | /r/programming | 2023-05-02

    1) Go support is right here https://github.com/200sc/bebop

  • bambam

    auto-generate capnproto schema from your golang source files. Depends on go-capnproto-1.0 at https://github.com/glycerine/go-capnproto

  • parco

    🏇🏻 generalist, fast and tiny binary parser and compiler generator, powered by Go 1.18+ Generics

  • polyglot

    A high-performance serialization framework used for encoding and decoding arbitrary datastructures across languages. (by loopholelabs)

  • bel

    Generate TypeScript interfaces from Go structs/interfaces - useful for JSON RPC

  • go-cardano-serialization

    Golang library for serialization & deserialization of Cardano data structures.

  • fpdecimal

    🛫 Fixed-Point Decimals

  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo
NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).

Go Serialization related posts

Index

What are some of the best open-source Serialization projects in Go? This list will help you:

Project Stars
1 jsoniter 13,076
2 goprotobuf 9,546
3 mapstructure 7,665
4 gogoprotobuf 5,629
5 msgpack 2,284
6 go-codec 1,817
7 hprose 1,256
8 go-capnp 1,152
9 csvutil 889
10 cbor 659
11 karmem 630
12 go-capnproto 288
13 watson 270
14 php_session_decoder 160
15 structomap 144
16 binstruct 89
17 bebop 67
18 bambam 65
19 parco 60
20 polyglot 37
21 bel 37
22 go-cardano-serialization 29
23 fpdecimal 28

Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com