Python Serialization

Open-source Python projects categorized as Serialization

Top 23 Python Serialization Projects

  • marshmallow

    A lightweight library for converting complex objects to and from simple Python datatypes.

    Project mention: Faster time-to-market with API-first | dev.to | 2022-10-25

    Uses a robust data validation library: validating payloads is a complex business. Your data validation library must handle optional and required properties, string formats like ISO dates and UUIDs (both dates and UUIDs are string types in OpenAPI), and strict vs loose type validation (should a string pass as an integer if it can be casted?). Also, in the case of Python, you need to make sure 1 and 0 don’t pass for True and False when it comes to boolean properties. In my experience, the best data validation libraries in the Python ecosystem are pydantic and marshmallow. From the above-mentioned libraries, flasgger and flask-smorest work with marshmallow.

  • orjson

    Fast, correct Python JSON library supporting dataclasses, datetimes, and numpy

    Project mention: Starlite development updates January ’23 | reddit.com/r/Python | 2023-01-10

    In version 1.45.0, we introduced msgspec as our serialization backend, replacing orjson. This had some immediate performance benefits, but that's not the main reason we made the switch.

  • InfluxDB

    Build time-series-based applications quickly and at scale.. InfluxDB is the Time Series Platform where developers build real-time applications for analytics, IoT and cloud-native services. Easy to start, it is available in the cloud or on-premises.

  • itsdangerous

    Safely pass trusted data to untrusted environments and back.

  • Schematics

    Python Data Structures for Humans™.

  • strictyaml

    Type-safe YAML parser and validator.

    Project mention: DO YOU YAML? | dev.to | 2023-01-16

    YAML stands for "YAML Ain’t Markup Language" - this is known as a recursive acronym. YAML is often used for writing configuration files. It’s human readable, easy to understand and can be used with other programming languages. Although YAML is commonly used in many disciplines, it has received criticism on the amoutn of whitespace .yml files have, difficulty in editing, and complexity of the standard. Despite the criticism, properly using YAML ensures that you can reproduce the results of a project and makes sure that the virtual environment packages play nicely with system packages. (If you're looking for another way to share environments there are other alternatives to YAML which include StrictYAML (a type-safe YAML parser) and NestedText)

  • jsonpickle

    Python library for serializing any arbitrary object graph into JSON. It can take almost any Python object and turn the object into JSON. Additionally, it can reconstitute the object back into Python.

    Project mention: For fun, I created a library to serialize / deserialize any python object into JSON | reddit.com/r/Python | 2022-08-27

    How it's different than jsonpickle?

  • serpy

    ridiculously fast object serialization

  • Sonar

    Write Clean Python Code. Always.. Sonar helps you commit clean code every time. With over 225 unique rules to find Python bugs, code smells & vulnerabilities, Sonar finds the issues while you focus on the work.

  • cattrs

    Complex custom class converters for attrs.

    Project mention: Noob question on saving objects in YAML files | reddit.com/r/learnpython | 2022-10-13

    That being said, data serialization is a very common thing to do, so naturally there are tons of libraries that automate it for you. Personally, using dataclasses and cattrs is my goto way for doing such things.

  • pysimdjson

    Python bindings for the simdjson project.

    Project mention: Processing JSON 2.5x faster than simdjson with msgspec | reddit.com/r/Python | 2022-10-03

    simdjson

  • PyLD

    JSON-LD processor written in Python

    Project mention: Making SEO better for blog posts with Structured Data | dev.to | 2023-01-22
  • msgspec

    A fast and friendly JSON/MessagePack library, with optional schema validation

    Project mention: Koda Validate: Alternative to Pydantic that is faster, more flexible, and async-compatible. | reddit.com/r/Python | 2023-01-26

    I maintain another "fast" validation-adjacent project (https://jcristharif.com/msgspec/) that handles only the serialization + validation component, not worrying about validation of internal function calls. I could see these projects pairing really nicely together, will definitely keep koda-validate in mind for future work.

  • mashumaro

    Fast and well tested serialization framework on top of dataclasses

  • colander

    A serialization/deserialization/validation library for strings, mappings and lists.

  • Fast JSON schema for Python

    Fast JSON schema validator for Python.

    Project mention: I wrote okjson - A fast, simple, and pythonic JSON Schema Validator | reddit.com/r/Python | 2022-03-31

    I had a requirement to process and validate large payloads of JSON concurrently for a web service, initially I implemented it using jsonschema and fastjsonschema but I found the whole JSON Schema Specification to be confusing at times and on top of that wanted better performance. Albeit there are ways to compile/cache the schema, I wanted to move away from the schema specification so I wrote a validation library inspired by the design of tiangolo/sqlmodel (type hints) to solve this problem easier.

  • mcap

    MCAP is a modular, performant, and serialization-agnostic container file format for pub/sub messages, primarily intended for use in robotics applications.

    Project mention: Evaluation of robotics data recording file formats | news.ycombinator.com | 2022-11-28
  • nestedtext

    Human Readable and Writable Data Interchange Format

    Project mention: The yaml document from hell | reddit.com/r/programming | 2023-01-12

    As /u/astatine said, an excellent but under-recognized alternative syntax for configuration files is NestedText, where everything is a string unless the ingesting code says otherwise, and there is no escaping needed ever.

  • jsons

    🐍 A Python lib for (de)serializing Python objects to/from JSON

  • pySHACL

    A Python validator for SHACL

  • cachew

    Transparent and persistent cache/serialization powered by type hints

  • hjson-py

    Hjson for Python

  • Trafaret

    Ultimate transformation library that supports validation, contexts and aiohttp.

  • apischema

    JSON (de)serialization, GraphQL and JSON schema generation using Python typing.

  • flatdata

    Write-once, read-many, minimal overhead binary structured file format.

    Project mention: How should I handle storing and reading from large amounts of data in my project? | reddit.com/r/rust | 2022-04-10

    Shameless self-plug: You could have a look at https://github.com/heremaps/flatdata to see if it fits your needs: Read-only memory mapped flat data structures providing fast random access.

  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2023-01-26.

Python Serialization related posts

Index

What are some of the best open-source Serialization projects in Python? This list will help you:

Project Stars
1 marshmallow 6,396
2 orjson 4,197
3 itsdangerous 2,618
4 Schematics 2,550
5 strictyaml 1,187
6 jsonpickle 1,097
7 serpy 916
8 cattrs 592
9 pysimdjson 564
10 PyLD 513
11 msgspec 481
12 mashumaro 472
13 colander 435
14 Fast JSON schema for Python 363
15 mcap 290
16 nestedtext 290
17 jsons 260
18 pySHACL 189
19 cachew 186
20 hjson-py 186
21 Trafaret 174
22 apischema 165
23 flatdata 158
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com