Open-source projects categorized as Serialization | Edit details

Top 23 Serialization Open-Source Projects

  • GitHub repo Protobuf

    Protocol Buffers - Google's data interchange format

    Project mention: I made an NBT-based data format, but a little more general purpose | reddit.com/r/gamedev | 2021-06-16

    Protocol Buffers -- compact encoding, lots of good tools, supports lots of languages

  • GitHub repo fastjson

    A fast JSON parser/generator for Java.

    Project mention: Ask HN: Differences in Programming Style by Nationalities | news.ycombinator.com | 2021-04-19

    I first noticed a difference in national styles when I contributed a patch to fastjson, a JSON parser in Java from Alibaba.

    Take a look around the repo; it'll look odd to a western programmer. There is a lot of duplication. And the tests don't have names, they are numbered.


    This was several years ago. I did end up choosing it since it really was the fastest.

  • GitHub repo FlatBuffers

    FlatBuffers: Memory Efficient Serialization Library

    Project mention: I made an NBT-based data format, but a little more general purpose | reddit.com/r/gamedev | 2021-06-16

    FlatBuffers -- encoding is not as compact as protocol buffers, but encoding / decoding is faster

  • GitHub repo store.js

    Cross-browser storage for all use cases, used across the web.

    Project mention: Deployed my first React project.. yes, a Pokedex. | reddit.com/r/reactjs | 2021-04-21

    and the pokemon user can catch is saved using session based local storage, Store js (https://github.com/marcuswestin/store.js)

  • GitHub repo jsoniter

    A high-performance 100% compatible drop-in replacement of "encoding/json" (by json-iterator)

    Project mention: OjG now has a tokenizer that is almost 10 times faster than json.Decode | reddit.com/r/golang | 2021-04-18

    Need to compare with https://github.com/json-iterator/go too!

  • GitHub repo Json.NET

    Json.NET is a popular high-performance JSON framework for .NET

    Project mention: Tutorial request: Parser | reddit.com/r/PowerShell | 2021-06-14

    If you want to look at how PowerShell does it, the source code for ConvertTo-Json is here and because it uses the Newtonsoft.Json library, that source code (which does the real work) ends up calling this Serialize() function

  • GitHub repo goprotobuf

    Go support for Google's protocol buffers (by golang)

    Project mention: A complete yet beginner friendly guide on how to secure Linux | reddit.com/r/linux | 2021-06-04
  • GitHub repo protobuf

    Protocol Buffers for JavaScript (& TypeScript).

    Project mention: Is Protobuf.js Faster Than JSON? | dev.to | 2021-04-21

    The Performance Section in protobufjs documentation led us to replace our SDK exporter from JSON to protobuf payload, thinking we will get better performance.

  • GitHub repo marshmallow

    A lightweight library for converting complex objects to and from simple Python datatypes.

    Project mention: What's the fastest way to parse JSON to output? | reddit.com/r/flask | 2021-02-21

    I was looking at https://github.com/marshmallow-code/marshmallow That's a nice library to use to parsing?

  • GitHub repo Kryo

    Java binary serialization and cloning: fast, efficient, automatic

    Project mention: Downsides to using sun.misc.unsafe for serialization (assuming the code is thoroughly-tested)? | reddit.com/r/java | 2021-03-13
  • GitHub repo gogoprotobuf

    [Looking for new ownership] Protocol Buffers for Go with Gadgets (by gogo)

    Project mention: Обережно кодогенерація | dev.to | 2020-09-21

    В проекті ми використовуємо офіційну бібліотеку Protobuf github.com/protocolbuffers/protobuf, яка підчас серіалізації використовує рефлексію і будує слайс байтів через append. А потім я дізнався про "Protocol Buffers for Go with Gadgets" github.com/gogo/protobuf, бібліотеку-fork яка генерує додатковий код щоб прибрати рефексію підчас серіалізації і вже записує в слайс байтів по індексу бо так швидше. Коли змінював одну бібліотеку на іншу то важливим вважав що стало працювати швидше і написані раніше тести пройшли успішно. І все б було гаразд але в проекті існувала латка яка через пару тижнів після заміни перезапустила мікросервіс через паніку:

  • GitHub repo mapstructure

    Go library for decoding generic map values into native Go structures and vice versa.

    Project mention: Implement login user API that returns PASETO or JWT access token in Go | dev.to | 2021-04-24

    First, the TokenSymmetricKey of type string. We have to specify the mapstructure tag for it because viper uses mapstructure package to parse the config data. Please refer to the lecture 12 of the course if you don’t know how to use viper.

  • GitHub repo ysoserial

    A proof-of-concept tool for generating payloads that exploit unsafe Java object deserialization.

    Project mention: Why do I think Haskell is a good choice in regards to Software Security? | dev.to | 2021-05-31

    A similar issue has occurred with Java (and other languages, see https://frohoff.github.io/appseccali-marshalling-pickles/). Java provided a suberbly user-friendly way of serializing any object to disk and recovering it back in its original form. The only unfortunate problem was that there was no way to say which object you are expecting! This allows attackers to send you objects that, upon deserialization in your program, become nasties that wreak havoc and steal data.

  • GitHub repo HandyJSON

    A handy swift json-object serialization/deserialization library (by alibaba)

  • GitHub repo class-transformer

    Decorator-based transformation, serialization, and deserialization between objects and classes.

    Project mention: [Typescript] Any elegant way to force object shape? | reddit.com/r/learnprogramming | 2021-06-10

    What you can do is use something like “class-transformer” library (https://github.com/typestack/class-transformer) or something like MakeTypes (https://jvilk.com/MakeTypes/). With the latest, you’d use the generated proxy classes and, since it comes as a node module, you can re-generate these whenever the PostBody changes (can be included in the build process to do it automatically).

  • GitHub repo Protobuf.NET

    Protocol Buffers library for idiomatic .NET

    Project mention: Integrating Apollo Studio with GraphQL for .NET - Part 2 | dev.to | 2021-05-28

    It's pretty straight-forward to follow the protobuf-net docs to serialize the report, but we should really GZIP the stream for sending to reduce bandwidth consumption and improve performance:

  • GitHub repo MessagePack for C# (.NET, .NET Core, Unity, Xamarin)

    Extremely Fast MessagePack Serializer for C#(.NET, .NET Core, Unity, Xamarin). / msgpack.org[C#]

    Project mention: Json locations | reddit.com/r/gamedev | 2021-05-05

    The solution is always use a data contract serializer. Json.NET can of course be configured as such. My favorite serializer is MessagePack.

  • GitHub repo ultrajson

    Ultra fast JSON decoder and encoder written in C with Python bindings

  • GitHub repo kotlinx.serialization

    Kotlin multiplatform / multi-format serialization

    Project mention: Kotlin Team AMA #3: Ask Us Anything | reddit.com/r/Kotlin | 2021-05-27

    The link is broken. It should be https://github.com/Kotlin/kotlinx.serialization/issues/1129

  • GitHub repo cereal

    A C++11 library for serialization

  • GitHub repo parse5

    HTML parsing/serialization toolset for Node.js. WHATWG HTML Living Standard (aka HTML5)-compliant.

    Project mention: Getting Started with Deno | dev.to | 2021-02-28

    After some googling, I landed on parse5 which appeared to have wide usage and offered a simple, low-level tree API at its core.

  • GitHub repo Schematics

    Python Data Structures for Humans™.

  • GitHub repo Bond

    Bond is a cross-platform framework for working with schematized data. It supports cross-language de/serialization and powerful generic mechanisms for efficiently manipulating data. Bond is broadly used at Microsoft in high scale services. (by microsoft)

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2021-06-16.


What are some of the best open-source Serialization projects? This list will help you:

Project Stars
1 Protobuf 48,309
2 fastjson 23,469
3 FlatBuffers 16,367
4 store.js 13,467
5 jsoniter 9,409
6 Json.NET 8,605
7 goprotobuf 7,712
8 protobuf 7,530
9 marshmallow 5,564
10 Kryo 5,060
11 gogoprotobuf 4,650
12 mapstructure 4,558
13 ysoserial 4,204
14 HandyJSON 3,761
15 class-transformer 3,629
16 Protobuf.NET 3,433
17 MessagePack for C# (.NET, .NET Core, Unity, Xamarin) 3,379
18 ultrajson 3,290
19 kotlinx.serialization 3,130
20 cereal 2,810
21 parse5 2,662
22 Schematics 2,423
23 Bond 2,306