Python Serialization

Open-source Python projects categorized as Serialization | Edit details

Top 23 Python Serialization Projects

  • GitHub repo marshmallow

    A lightweight library for converting complex objects to and from simple Python datatypes.

    Project mention: The Pocket Guide To API Request Validation You Wish You Had Earlier | dev.to | 2022-01-06

    Marshmallow

  • GitHub repo orjson

    Fast, correct Python JSON library supporting dataclasses, datetimes, and numpy

    Project mention: A simple Python package written using Rust and Pyo3 (XBRL XML to JSON/CSV parser) | reddit.com/r/Python | 2022-01-11

    I recently discovered the amazing Pyo3 library which makes it relatively easy to write a Python package using a Rust library. This enables the development of Python packages that utilize the blazing speed of Rust with the usability of Python. Packages such as ORJSON use this approach to achieve state-of-the-art speeds through a Python package.

  • Scout APM

    Less time debugging, more time building. Scout APM allows you to find and fix performance issues with no hassle. Now with error monitoring and external services monitoring, Scout is a developer's best friend when it comes to application development.

  • GitHub repo Schematics

    Python Data Structures for Humans™.

  • GitHub repo itsdangerous

    Safely pass trusted data to untrusted environments and back.

    Project mention: Flask 2.0 is coming, please help us test | reddit.com/r/Python | 2021-04-16

    This major release of Flask is accompanied by major releases of Werkzeug, Jinja2, click, and itsdangerous which we'd also welcome and appreciate testing (their pre releases are installed with the Flask pre release).

  • GitHub repo strictyaml

    Type-safe YAML parser and validator.

    Project mention: how to build a JSON parser? | reddit.com/r/C_Programming | 2021-12-26

    Don't say "it's just about the quality of the implementing team".

  • GitHub repo serpy

    ridiculously fast object serialization

  • GitHub repo KQ

    Kafka-based Job Queue for Python

  • SonarQube

    Static code analysis for 29 languages.. Your projects are multi-language. So is SonarQube analysis. Find Bugs, Vulnerabilities, Security Hotspots, and Code Smells so you can release quality code every time. Get started analyzing your projects today for free.

  • GitHub repo pysimdjson

    Python bindings for the simdjson project.

    Project mention: How I cut GTA Online loading times by 70% | reddit.com/r/programming | 2021-02-28

    I don't think JSON is really the problem - parsing 10MB of JSON is not so slow. For example, using Python's json.load takes about 800ms for a 47MB file on my system, using something like simdjson cuts that down to ~70ms.

  • GitHub repo PyLD

    JSON-LD processor written in Python

    Project mention: What is Structured Data? | dev.to | 2021-10-29

    Web pages are an interesting example of both structured and unstructured data. There are specific elements one could look at for certain information like the element or other semantic elements like or . The problem though is that these elements are more like our "address" example earlier - they often contain more than just the strict data we are looking for. A title might have a prefix or suffix of the website's name. An article or section might have many other layers of

    , or any other elements to help form the site's structure. To top it off, the HTML structure can vary wildly from site to site. If you were wanting to extract data from multiple websites, it can get very hard very fast.

    That said, there are a number of ways to embed structured data into web pages. A web page could use Microdata, RDFa, JSON-LD or Open Graph to express structured data. More than that though, a web page can use multiple of these at the same time. Open Graph is commonly used as a method of defining details for a link preview while the others might express more complex data like product pricing or reviews.

    Having standard formats like Microdata or JSON-LD are a good start but only represent the format of the data - we need a common vocabulary so we can understand the data those formats encode. One common vocabulary used is called Schema.org and provides over 700 types including types to describe people, places, products, recipes, reviews, vehicles, movies and medical devices. Using Schema.org for structured data on a website can help search engines provide richer experiences in the search results.

    Summary

    Structured data, through standardising expected properties and value formats, makes the sharing and processing of data easier. Web pages in particular benefit from encoding structured data in their mark-up where it can be used by search engines and other tools.

  • GitHub repo colander

    A serialization/deserialization/validation library for strings, mappings and lists.

  • GitHub repo cattrs

    Complex custom class converters for attrs.

    Project mention: I use attrs instead of pydantic | news.ycombinator.com | 2021-08-26

    ```

    Cattrs has some problems with generics [1] [2]. Dacite and marshmallow-dataclasses don't support generics well either, with some issues around Union types.

    They do work well for simple python types but what I'd like to see is guarantee that the serialisation operation is completely reversible and if not raise warning/exception.

    [1] https://github.com/Tinche/cattrs/issues/149

  • GitHub repo mashumaro

    Fast and well tested serialization framework on top of dataclasses

    Project mention: I use attrs instead of pydantic | news.ycombinator.com | 2021-08-26

    Having run into these issues with Pydantic, we've been using Mashumaro[1], which, while not having all the bells and whistles of Pydantic, has served us pretty well.

    1: https://github.com/Fatal1ty/mashumaro

  • GitHub repo Fast JSON schema for Python

    Fast JSON schema validator for Python.

  • GitHub repo jsons

    🐍 A Python lib for (de)serializing Python objects to/from JSON

  • GitHub repo cachew

    Transparent and persistent cache/serialization powered by type hints

    Project mention: How I collect and use 50 sources of my personal data | news.ycombinator.com | 2021-02-06

    Yep! In fact I've tried interoperating with Datasette (e.g. shared here https://news.ycombinator.com/item?id=25090643 )

    One secret sauce is using 'automatic' caching of data in sqlite -- this allows both for faster access and having an additional interface for the data as a collateral https://github.com/karlicoss/cachew#readme

    Still need to polish this a bit, but ultimately hoping to properly plug into Datasette, I'm impressed by its data exploration capabilities!

  • GitHub repo hjson-py

    Hjson for Python

  • GitHub repo Trafaret

    Ultimate transformation library that supports validation, contexts and aiohttp.

  • GitHub repo pySHACL

    A Python validator for SHACL

    Project mention: Python tools for the semantic web | dev.to | 2021-02-15

    pySHACL is a pure Python module which allows for the validation of RDF graphs against Shapes Constraint Language (SHACL) graphs. It is built on top of rdflib.

  • GitHub repo desert

    Deserialize to objects while staying DRY (by python-desert)

  • GitHub repo dataconf

    Simple dataclasses configuration management for Python with hocon/json/yaml/properties/env-vars/dict support.

    Project mention: Show HN: Dataconf, Python dataclasses config (hocon/JSON/YAML/env-vars/dict) | news.ycombinator.com | 2021-11-07
  • GitHub repo lupin is a Python JSON object mapper

    Python document object mapper (load python object from JSON and vice-versa)

  • GitHub repo Maat

    Validation and transformation library powered by deductive ascending parser. Made to be extended for any kind of project.

    Project mention: Parse, Don't Validate (2019) | news.ycombinator.com | 2021-06-26

    https://github.com/Attumm/Maat/blob/main/tests/test_validati...

    Since validation is written as dictionaries its possible to store the validations in caching db such as Redis.

    And since its simple its easy to extend for anyone use case.

    Benchmarks of pydantic has Maat around twice as Pydantic.

  • GitHub repo restframework-serializer-permissions

    Drop in replacement for Django Restframework Serializers to add permission based field serialization.

    Project mention: Restframework-Serializer-Permissions | news.ycombinator.com | 2021-04-06
NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2022-01-11.

Python Serialization related posts

Index

What are some of the best open-source Serialization projects in Python? This list will help you:

Project Stars
1 marshmallow 5,888
2 orjson 2,736
3 Schematics 2,465
4 itsdangerous 2,376
5 strictyaml 964
6 serpy 885
7 KQ 538
8 pysimdjson 506
9 PyLD 467
10 colander 413
11 cattrs 398
12 mashumaro 362
13 Fast JSON schema for Python 322
14 jsons 212
15 cachew 170
16 hjson-py 167
17 Trafaret 165
18 pySHACL 146
19 desert 104
20 dataconf 34
21 lupin is a Python JSON object mapper 19
22 Maat 18
23 restframework-serializer-permissions 7
Find remote jobs at our new job board 99remotejobs.com. There are 29 new remote jobs listed recently.
Are you hiring? Post a new remote job listing for free.
OPS - Build and Run Open Source Unikernels
Quickly and easily build and deploy open source unikernels in tens of seconds. Deploy in any language to any cloud.
github.com/nanovms