InfluxDB 3 OSS is now GA. Transform, enrich, and act on time series data directly in the database. Automate critical tasks and eliminate the need to move data externally. Download now. Learn more →
Top 23 Python parallel-computing Projects
-
-
InfluxDB
InfluxDB – Built for High-Performance Time Series Workloads. InfluxDB 3 OSS is now GA. Transform, enrich, and act on time series data directly in the database. Automate critical tasks and eliminate the need to move data externally. Download now.
-
swifter
A package which efficiently applies any function to a pandas dataframe or series in the fastest available manner (by jmcarpenter2)
-
pyper – Concurrent Python made simple
-
-
-
Hyperactive
An optimization and data collection toolbox for convenient and fast prototyping of computationally expensive models.
-
mpi4jax
Zero-copy MPI communication of JAX arrays, for turbo-charged HPC applications in Python :zap:
-
Sevalla
Deploy and host your apps and databases, now with $50 credit! Sevalla is the PaaS you have been looking for! Advanced deployment pipelines, usage-based pricing, preview apps, templates, human support by developers, and much more!
-
-
pyhpc-benchmarks
A suite of benchmarks for CPU and GPU performance of the most popular high-performance libraries for Python :rocket:
-
Project mention: Polars Cloud: The Distributed Cloud Architecture to Run Polars Anywhere | news.ycombinator.com | 2025-03-07
Bodo is a JIT compiler for Pandas/Numpy code that statically types the code during compilation time. However, it infers types automatically (looks at file metadata etc) and not really built for manually typing everything.
https://github.com/bodo-ai/Bodo
-
-
Fast-Poisson-Image-Editing
A fast poisson image editing implementation that can utilize multi-core CPU or GPU to handle a high-resolution image input.
-
PSyclone
PSyclone is a source-to-source Fortran compiler designed to programmatically optimise, parallelise and instrument HPC applications via user-provided transformation scripts.
-
-
-
-
mlToolKits
learningOrchestra is a distributed Machine Learning integration tool that facilitates and streamlines iterative processes in a Data Science project.
-
b-rabbit
A thread safe library that aims to provide a simple API for interfacing with RabbitMQ. Built on top of rabbitpy, the library make it very easy to use the RabbitMQ message broker with just few lines of code. It implements all messaging pattern used by message brokers
-
-
-
YellowPage-scraper
A YellowPage scraper is a Python program/script that extracts data from the YellowPages.com website using the Python programming language. The scraper can be used to gather information such as business names, addresses, phone numbers, emails and reviews from the YellowPages website.
-
-
parallel-book-generation
Parallel AI Book Generation Architecture - Generate complete books in under 5 minutes instead of 60-90 minutes using stateless parallel chapter generation with AWS Step Functions. A fundamental rethinking of how AI content generation should work at scale.
Project mention: Parallel book generation in 3.5 minutes instead of 60 (stateless architecture) | news.ycombinator.com | 2025-07-17- Quality: Professional editor rated output 4/5
I've open sourced the architecture pattern: https://github.com/Traviseric/parallel-book-generation
Also built a "knobs" framework as one example of how I'm maintaining quality/variation in parallel generation: https://github.com/Traviseric/knobs-framework
Currently using this to generate 100 books in 90 days as a real-world test. Happy to answer questions about the technical challenges or parallel content generation in general.
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
Python parallel-computing discussion
Python parallel-computing related posts
-
Minimal implementation of Mamba, the new LLM architecture, in 1 file of PyTorch
-
I made a Python package to do adaptive learning of functions in parallel [P]
-
I made a Python package to do adaptive sampling of functions in parallel [OC]
-
I made a Python package to do adaptive sampling of functions in parallel [OC]
-
I made a Python package to do adaptive sampling of functions in parallel [OC]
-
Introducing Markdown Code Runner: Automatically execute code blocks in your Markdown files! 🚀
-
Does Julia have something like pythons adaptive?
-
A note from our sponsor - InfluxDB
www.influxdata.com | 1 Sep 2025
Index
What are some of the best open-source parallel-computing projects in Python? This list will help you:
| # | Project | Stars |
|---|---|---|
| 1 | Joblib | 4,192 |
| 2 | swifter | 2,616 |
| 3 | pyper | 1,465 |
| 4 | adaptive | 1,200 |
| 5 | pyopencl | 1,111 |
| 6 | Hyperactive | 529 |
| 7 | mpi4jax | 491 |
| 8 | jug | 459 |
| 9 | pyhpc-benchmarks | 328 |
| 10 | Bodo | 300 |
| 11 | pqdm | 295 |
| 12 | Fast-Poisson-Image-Editing | 275 |
| 13 | PSyclone | 120 |
| 14 | nnscaler | 115 |
| 15 | python-performance | 103 |
| 16 | wrapyfi | 77 |
| 17 | mlToolKits | 76 |
| 18 | b-rabbit | 31 |
| 19 | RayEnvWrapper | 23 |
| 20 | lava | 17 |
| 21 | YellowPage-scraper | 10 |
| 22 | blazer | 7 |
| 23 | parallel-book-generation | 2 |
Sponsored
InfluxDB – Built for High-Performance Time Series Workloads
InfluxDB 3 OSS is now GA. Transform, enrich, and act on time series data directly in the database. Automate critical tasks and eliminate the need to move data externally. Download now.
www.influxdata.com