Go Serialization

Open-source Go projects categorized as Serialization

Top 23 Go Serialization Projects

  • jsoniter

    A high-performance 100% compatible drop-in replacement of "encoding/json" (by json-iterator)

  • Project mention: Handling high-traffic HTTP requests with JSON payloads | /r/golang | 2023-12-07

    Since most of the time would be spent decoding json, you could try to cut this time using https://github.com/bytedance/sonic or https://github.com/json-iterator/go, both are drop-in replacements for the stdlib, sonic is faster.

  • goprotobuf

    Go support for Google's protocol buffers (by golang)

  • Project mention: Protoc Plugins with Go | dev.to | 2023-08-19

    Now let’s take a look at the source code of the protoc-gen-go plugin:

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo
  • mapstructure

    Go library for decoding generic map values into native Go structures and vice versa.

  • Project mention: How do I marshal a JSON array into a map? | /r/golang | 2023-12-07
  • gogoprotobuf

    [Deprecated] Protocol Buffers for Go with Gadgets (by gogo)

  • msgpack

    msgpack.org[Go] MessagePack encoding for Golang (by vmihailenco)

  • Project mention: Rob Pike: Gobs of data (2011) | news.ycombinator.com | 2023-12-04

    Someone made a benchmark of serialization libraries in go [1], and I was surprised to see gobs is one of the slowest ones, specially for decoding. I suspect part of the reason is that the API doesn't not allow reusing decoders [2]. From my explorations it seems like both JSON [3], message-pack [4] and CBOR [5] are better alternatives.

    By the way, in Go there are a like a million JSON encoders because a lot of things in the std library are not really coded for maximum performance but more for easy of usage, it seems. Perhaps this is the right balance for certain things (ex: the http library, see [6]).

    There are also a bunch of libraries that allow you to modify a JSON file "in place", without having to fully deserialize into structs (ex: GJSON/SJSON [7] [8]). This sounds very convenient and more efficient that fully de/serializing if we just need to change the data a little.

    --

    1: https://github.com/alecthomas/go_serialization_benchmarks

    2: https://github.com/golang/go/issues/29766#issuecomment-45492...

    --

    3: https://github.com/goccy/go-json

    4: https://github.com/vmihailenco/msgpack

    5: https://github.com/fxamacker/cbor

    --

    6: https://github.com/valyala/fasthttp#faq

    --

    7: https://github.com/tidwall/gjson

    8: https://github.com/tidwall/sjson

  • go-codec

    idiomatic codec and rpc lib for msgpack, cbor, json, etc. msgpack.org[Go] (by ugorji)

  • hprose

    Hprose is a cross-language RPC. This project is Hprose for Golang.

  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo
  • go-capnp

    Cap'n Proto library and code generator for Go

  • Project mention: Sandstorm: Open-source platform for self-hosting web app | news.ycombinator.com | 2023-06-04

    The Go implementation of capnp is alive and well, too. I highly recommend it: https://github.com/capnproto/go-capnp

  • csvutil

    csvutil provides fast and idiomatic mapping between CSV and Go (golang) values.

  • cbor

    CBOR codec (RFC 8949) with CBOR tags, Go struct tags (toarray, keyasint, omitempty), float64/32/16, big.Int, and fuzz tested billions of execs.

  • Project mention: Rob Pike: Gobs of data (2011) | news.ycombinator.com | 2023-12-04

    Someone made a benchmark of serialization libraries in go [1], and I was surprised to see gobs is one of the slowest ones, specially for decoding. I suspect part of the reason is that the API doesn't not allow reusing decoders [2]. From my explorations it seems like both JSON [3], message-pack [4] and CBOR [5] are better alternatives.

    By the way, in Go there are a like a million JSON encoders because a lot of things in the std library are not really coded for maximum performance but more for easy of usage, it seems. Perhaps this is the right balance for certain things (ex: the http library, see [6]).

    There are also a bunch of libraries that allow you to modify a JSON file "in place", without having to fully deserialize into structs (ex: GJSON/SJSON [7] [8]). This sounds very convenient and more efficient that fully de/serializing if we just need to change the data a little.

    --

    1: https://github.com/alecthomas/go_serialization_benchmarks

    2: https://github.com/golang/go/issues/29766#issuecomment-45492...

    --

    3: https://github.com/goccy/go-json

    4: https://github.com/vmihailenco/msgpack

    5: https://github.com/fxamacker/cbor

    --

    6: https://github.com/valyala/fasthttp#faq

    --

    7: https://github.com/tidwall/gjson

    8: https://github.com/tidwall/sjson

  • karmem

    Karmem is a fast binary serialization format, faster than Google Flatbuffers and optimized for TinyGo and WASM.

  • go-capnproto

    Cap'n Proto library and parser for go. This is go-capnproto-1.0, and does not have rpc. See https://github.com/zombiezen/go-capnproto2 for 2.0 which has rpc and capabilities.

  • watson

    WATSON: Wasted but Amazing Turing-incomplete Stack-based Object Notation (by genkami)

  • php_session_decoder

    PHP session encoder/decoder written in Go

  • structomap

    Easily and dynamically generate maps from Go static structures

  • binstruct

    Golang binary decoder for mapping data into the structure

  • bebop

    bebop wire format in Go (by 200sc)

  • bambam

    auto-generate capnproto schema from your golang source files. Depends on go-capnproto-1.0 at https://github.com/glycerine/go-capnproto

  • mus-go

    MUS format serializer with validation support for Golang

  • Project mention: Serializing complex data structures with mus-go | /r/golang | 2023-10-31

    Now, thanks to the pm package, mus-go can serialize data structures such as a graph or linked list.

  • parco

    πŸ‡πŸ» generalist, fast and tiny binary parser and compiler generator, powered by Go 1.18+ Generics

  • polyglot

    A high-performance serialization framework used for encoding and decoding arbitrary datastructures across languages. (by loopholelabs)

  • bel

    Generate TypeScript interfaces from Go structs/interfaces - useful for JSON RPC

  • go-cardano-serialization

    Golang library for serialization & deserialization of Cardano data structures.

  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo
NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).

Go Serialization related posts

  • Handling high-traffic HTTP requests with JSON payloads

    5 projects | /r/golang | 7 Dec 2023
  • How do I marshal a JSON array into a map?

    1 project | /r/golang | 7 Dec 2023
  • Is there any equivalent to pydantic, serde, etc?

    8 projects | /r/golang | 6 Dec 2023
  • Rob Pike: Gobs of data (2011)

    10 projects | news.ycombinator.com | 4 Dec 2023
  • A Journey building a fast JSON parser and full JSONPath

    5 projects | news.ycombinator.com | 11 Oct 2023
  • Protoc Plugins with Go

    2 projects | dev.to | 19 Aug 2023
  • Struggling to get JSON response data into usable struct

    2 projects | /r/golang | 6 Jul 2023
  • A note from our sponsor - SaaSHub
    www.saashub.com | 24 May 2024
    SaaSHub helps you find the best software and product alternatives Learn more β†’

Index

What are some of the best open-source Serialization projects in Go? This list will help you:

Project Stars
1 jsoniter 13,108
2 goprotobuf 9,581
3 mapstructure 7,727
4 gogoprotobuf 5,637
5 msgpack 2,300
6 go-codec 1,822
7 hprose 1,254
8 go-capnp 1,158
9 csvutil 895
10 cbor 669
11 karmem 630
12 go-capnproto 288
13 watson 270
14 php_session_decoder 161
15 structomap 144
16 binstruct 90
17 bebop 67
18 bambam 65
19 mus-go 63
20 parco 60
21 polyglot 39
22 bel 37
23 go-cardano-serialization 29

Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com