JSON in data science projects: tips & tricks

• orjson

  19 5,676 8.4 Python

  Fast, correct Python JSON library supporting dataclasses, datetimes, and numpy

orjson is the fastest JSON library available for python. It natively manages dataclass objects, datetime, numpy and UUID objects.

• NumPy

  273 26,625 10.0 Python

  The fundamental package for scientific computing with Python.

  

Data science projects often use numpy. However, numpy objects are not JSON-serializable and therefore require conversion to standard python objects in order to be saved:

• Scout Monitoring

  www.scoutapm.com featured

  Free Django app performance insights with Scout Monitoring. Get Scout setup in minutes, and let us sweat the small stuff. A couple lines in settings.py is all you need to start monitoring your apps. Sign up for our free tier today.

  

• jsonlines

  6 119 3.9 CSS

  Documentation for the JSON Lines text file format

This can be remedied by using the [JSON Lines] format (https://jsonlines.org/). This involves nothing more and nothing less than placing one JSON object per line, so that you can browse the objects without having to parse the entire collection all at once.

Suggest a related project