Top 3 Python parallel-processing Projects

p_tqdm

1 438 0.0 Python

Parallel processing with progress bars

Project mention: Need Help With Multiprocessing or Multithreading or whatever it's called | /r/learnpython | 2023-06-24

I avoid dealing directly with multiprocessing and asyncio unless absolutely necessary. I find I can parallelize a lot of tasks simply with the p_tqdm library which abstracts a lot of that away. Just be careful with it as it makes copies of anything you feed into it to get around the GIL.

pyDag

2 24 0.0 Python

Scheduling Big Data Workloads and Data Pipelines in the Cloud with pyDag
InfluxDB

www.influxdata.com sponsored

Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
librer

1 11 9.3 Python

A cross-platform GUI file cataloging program with extensive customization options to suit user preferences. Highly optimized for multi-core parallel search speed, data integrity, and repository portability.

Project mention: Librer – a highly customizable X-platform file cataloguing program | news.ycombinator.com | 2024-02-20

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).