InfluxDB is the Time Series Platform where developers build real-time applications for analytics, IoT and cloud-native services. Easy to start, it is available in the cloud or on-premises. Learn more →
Top 23 Python Serialization Projects
-
marshmallow
A lightweight library for converting complex objects to and from simple Python datatypes.
Uses a robust data validation library: validating payloads is a complex business. Your data validation library must handle optional and required properties, string formats like ISO dates and UUIDs (both dates and UUIDs are string types in OpenAPI), and strict vs loose type validation (should a string pass as an integer if it can be casted?). Also, in the case of Python, you need to make sure 1 and 0 don’t pass for True and False when it comes to boolean properties. In my experience, the best data validation libraries in the Python ecosystem are pydantic and marshmallow. From the above-mentioned libraries, flasgger and flask-smorest work with marshmallow.
-
In version 1.45.0, we introduced msgspec as our serialization backend, replacing orjson. This had some immediate performance benefits, but that's not the main reason we made the switch.
-
InfluxDB
Build time-series-based applications quickly and at scale.. InfluxDB is the Time Series Platform where developers build real-time applications for analytics, IoT and cloud-native services. Easy to start, it is available in the cloud or on-premises.
-
-
-
YAML stands for "YAML Ain’t Markup Language" - this is known as a recursive acronym. YAML is often used for writing configuration files. It’s human readable, easy to understand and can be used with other programming languages. Although YAML is commonly used in many disciplines, it has received criticism on the amoutn of whitespace .yml files have, difficulty in editing, and complexity of the standard. Despite the criticism, properly using YAML ensures that you can reproduce the results of a project and makes sure that the virtual environment packages play nicely with system packages. (If you're looking for another way to share environments there are other alternatives to YAML which include StrictYAML (a type-safe YAML parser) and NestedText)
-
jsonpickle
Python library for serializing any arbitrary object graph into JSON. It can take almost any Python object and turn the object into JSON. Additionally, it can reconstitute the object back into Python.
Project mention: For fun, I created a library to serialize / deserialize any python object into JSON | reddit.com/r/Python | 2022-08-27How it's different than jsonpickle?
-
-
Sonar
Write Clean Python Code. Always.. Sonar helps you commit clean code every time. With over 225 unique rules to find Python bugs, code smells & vulnerabilities, Sonar finds the issues while you focus on the work.
-
Project mention: Noob question on saving objects in YAML files | reddit.com/r/learnpython | 2022-10-13
That being said, data serialization is a very common thing to do, so naturally there are tons of libraries that automate it for you. Personally, using dataclasses and cattrs is my goto way for doing such things.
-
Project mention: Processing JSON 2.5x faster than simdjson with msgspec | reddit.com/r/Python | 2022-10-03
simdjson
-
-
Project mention: Koda Validate: Alternative to Pydantic that is faster, more flexible, and async-compatible. | reddit.com/r/Python | 2023-01-26
I maintain another "fast" validation-adjacent project (https://jcristharif.com/msgspec/) that handles only the serialization + validation component, not worrying about validation of internal function calls. I could see these projects pairing really nicely together, will definitely keep koda-validate in mind for future work.
-
-
-
Project mention: I wrote okjson - A fast, simple, and pythonic JSON Schema Validator | reddit.com/r/Python | 2022-03-31
I had a requirement to process and validate large payloads of JSON concurrently for a web service, initially I implemented it using jsonschema and fastjsonschema but I found the whole JSON Schema Specification to be confusing at times and on top of that wanted better performance. Albeit there are ways to compile/cache the schema, I wanted to move away from the schema specification so I wrote a validation library inspired by the design of tiangolo/sqlmodel (type hints) to solve this problem easier.
-
mcap
MCAP is a modular, performant, and serialization-agnostic container file format for pub/sub messages, primarily intended for use in robotics applications.
Project mention: Evaluation of robotics data recording file formats | news.ycombinator.com | 2022-11-28 -
As /u/astatine said, an excellent but under-recognized alternative syntax for configuration files is NestedText, where everything is a string unless the ingesting code says otherwise, and there is no escaping needed ever.
-
-
-
-
-
-
-
Project mention: How should I handle storing and reading from large amounts of data in my project? | reddit.com/r/rust | 2022-04-10
Shameless self-plug: You could have a look at https://github.com/heremaps/flatdata to see if it fits your needs: Read-only memory mapped flat data structures providing fast random access.
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
The latest post mention was on 2023-01-26.
Python Serialization related posts
- Koda Validate: Alternative to Pydantic that is faster, more flexible, and async-compatible.
- Making SEO better for blog posts with Structured Data
- Parsing and validating `pyproject.toml` files with msgspec
- Parsing & validating pyproject.toml with msgspec
- Starlite development updates January ’23
- Beginners Guide to Yoast SEO 2023
- Msgspec: A fast and friendly serialization library for Python
-
A note from our sponsor - InfluxDB
www.influxdata.com | 1 Feb 2023
Index
What are some of the best open-source Serialization projects in Python? This list will help you:
| Project | Stars | |
|---|---|---|
| 1 | marshmallow | 6,396 |
| 2 | orjson | 4,197 |
| 3 | itsdangerous | 2,618 |
| 4 | Schematics | 2,550 |
| 5 | strictyaml | 1,187 |
| 6 | jsonpickle | 1,097 |
| 7 | serpy | 916 |
| 8 | cattrs | 592 |
| 9 | pysimdjson | 564 |
| 10 | PyLD | 513 |
| 11 | msgspec | 481 |
| 12 | mashumaro | 472 |
| 13 | colander | 435 |
| 14 | Fast JSON schema for Python | 363 |
| 15 | mcap | 290 |
| 16 | nestedtext | 290 |
| 17 | jsons | 260 |
| 18 | pySHACL | 189 |
| 19 | cachew | 186 |
| 20 | hjson-py | 186 |
| 21 | Trafaret | 174 |
| 22 | apischema | 165 |
| 23 | flatdata | 158 |
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com