Processing JSON 2.5x faster than simdjson with msgspec

This page summarizes the projects mentioned and recommended in the original post on /r/Python

InfluxDB - Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com
featured
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com
featured
  • ultrajson

    Ultra fast JSON decoder and encoder written in C with Python bindings

  • ujson

  • orjson

    Fast, correct Python JSON library supporting dataclasses, datetimes, and numpy

  • orjson

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo
  • pysimdjson

    Python bindings for the simdjson project.

  • simdjson

  • msgspec

    A fast serialization and validation library, with builtin support for JSON, MessagePack, YAML, and TOML

  • Msgspec achieves this performance by doing less work - it's only parsing the fields that are used for the query. Allocating objects in Python can be slow, by specifying the required fields for the query (though a type annotated schema), we reduce allocations to the bare minimum resulting in measurable speedups. There's a longer writeup in the example docs if you're interested.

  • conda

    A system-level, binary package and environment manager running on all major operating systems and platforms.

  • I wrote this up as a demo of some potential optimization work in conda, but the same principle could be applied to other projects.

  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo
NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts

  • Pyserde

    1 project | news.ycombinator.com | 6 May 2024
  • pyserde: Serialization library on top of dataclasses, inspired by serde-rs

    1 project | news.ycombinator.com | 18 Nov 2023
  • Need help developing a high performance Redis ORM for Python

    2 projects | /r/Python | 23 May 2023
  • Pyserde: Serialization library on top of dataclasses, inspired by serde-rs

    1 project | news.ycombinator.com | 21 May 2023
  • Blog post: Writing Python like it’s Rust

    2 projects | /r/Python | 20 May 2023