Processing JSON 2.5x faster than simdjson with msgspec

This page summarizes the projects mentioned and recommended in the original post on reddit.com/r/Python

Our great sponsors
  • InfluxDB - Access the most powerful time series database as a service
  • Sonar - Write Clean Python Code. Always.
  • SaaSHub - Software Alternatives and Reviews
  • ultrajson

    Ultra fast JSON decoder and encoder written in C with Python bindings

    ujson

  • orjson

    Fast, correct Python JSON library supporting dataclasses, datetimes, and numpy

    orjson

  • InfluxDB

    Access the most powerful time series database as a service. Ingest, store, & analyze all types of time series data in a fully-managed, purpose-built database. Keep data forever with low-cost storage and superior data compression.

  • pysimdjson

    Python bindings for the simdjson project.

    simdjson

  • msgspec

    A fast serialization and validation library, with builtin support for JSON, MessagePack, YAML, and TOML

    Msgspec achieves this performance by doing less work - it's only parsing the fields that are used for the query. Allocating objects in Python can be slow, by specifying the required fields for the query (though a type annotated schema), we reduce allocations to the bare minimum resulting in measurable speedups. There's a longer writeup in the example docs if you're interested.

  • conda

    A system-level, binary package and environment manager running on all major operating systems and platforms.

    I wrote this up as a demo of some potential optimization work in conda, but the same principle could be applied to other projects.

  • Sonar

    Write Clean Python Code. Always.. Sonar helps you commit clean code every time. With over 225 unique rules to find Python bugs, code smells & vulnerabilities, Sonar finds the issues while you focus on the work.

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts