Python Multiprocessing

Open-source Python projects categorized as Multiprocessing

Top 23 Python Multiprocessing Projects

  • Joblib

    Computing with Python functions.

  • PSpider


  • Mergify

    Updating dependencies is time-consuming.. Solutions like Dependabot or Renovate update but don't merge dependencies. You need to do it manually while it could be fully automated! Add a Merge Queue to your workflow and stop caring about PR management & merging. Try Mergify for free.

  • aiomultiprocess

    Take a modern Python codebase to the next level of performance.

  • imgp

    :camera_flash: Fast batch image resizer and rotator

  • Plutus

    An automated bitcoin wallet collider that brute forces random wallet addresses (by Isaacdelly)

  • crypto-rl

    Deep Reinforcement Learning toolkit: record and replay cryptocurrency limit order book data & train a DDQN agent

  • pytorch-A3C

    Simple A3C implementation with pytorch + multiprocessing

  • Sonar

    Write Clean Python Code. Always.. Sonar helps you commit clean code every time. With over 225 unique rules to find Python bugs, code smells & vulnerabilities, Sonar finds the issues while you focus on the work.

  • vermin

    Concurrently detect the minimum Python versions needed to run code - Vermin 1.6 will end support for running via Python 2.7. Python 3.x is going to be required but detection of 2.x functionality will remain functional.

    Project mention: Vermin 1.5.2 released | /r/madeinpython | 2023-06-17
  • lithops

    A multi-cloud framework for big data analytics and embarrassingly parallel jobs, that provides an universal API for building parallel applications in the cloud ☁️🚀

    Project mention: Lithops: A multi-cloud framework for embarrassingly parallel jobs | | 2023-01-14
  • think-async

    🌿 Exploring cooperative concurrency primitives in Python

  • multitasking

    Non-blocking Python methods using decorators

    Project mention: I made autothread: the easiest way to add parallelization to your code | /r/Python | 2022-12-22

    Oh cool, I missed that one when I did my initial research. I did come across multitasking, which is also similar but doesn't make it easy to return objects from the function.

  • lox

    Threading and Multiprocessing made easy.

    Project mention: Mpire: A Python package for easier and faster multiprocessing | | 2023-08-11

    I often use lox for this sort of thing. It can use threads or processes, and has a very ergonomic api.

  • ruck

    🧬 Modularised Evolutionary Algorithms For Python with Optional JIT and Multiprocessing (Ray) support. Inspired by PyTorch Lightning

  • NotEnoughAV1Encodes-Qt

    Linux GUI for AV1 Encoders

    Project mention: did I fail at preserving dvds? mpeg2 vs h264 vs av1 | /r/PleX | 2022-11-08

    I like using NotEnoughAV1Encodes. There's also Av1an, and ab-av1.

  • google_drive_ocr

    Perform OCR using Google's Drive API v3

  • geventmp

    GeventMP - Gevent Multiprocessing Extension

  • regta

    📅 Production-ready scheduler with async, multithreading and multiprocessing support for Python

    Project mention: [Automation] Scheduling Python Programs: Pushing Notifications, Executing SQLs, etc... | | 2023-09-24

    If your automation needs are focused on Python exclusively, Regta emerges as a compelling option, offering a wealth of Python-specific optimizations. Give it a try, and feel free to share your thoughts in the comments 🙌

  • gibbs

    Scale your ML workers asynchronously across processes and machines

  • mnqueues

    Monitored Multiprocessing Queues

  • treequeues

    High performance queues for pytree objects.

  • RemoteShell

    This is a multi-client to server cross platform remote shell written in Python using threading and multiprocessing.

  • autowpscan

    Assistant work tool for wpscan.

    Project mention: autowpscan | /r/cybersecurity | 2023-04-14
  • mpxl2csv

    Project mention: Convert Excel to csv using Python multiprocessing | /r/Python | 2023-04-27

    Source Code:

  • InfluxDB

    Collect and Analyze Billions of Data Points in Real Time. Manage all types of time series data in a single, purpose-built database. Run at any scale in any environment in the cloud, on-premises, or at the edge.

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2023-09-24.

Python Multiprocessing related posts


What are some of the best open-source Multiprocessing projects in Python? This list will help you:

Project Stars
1 Joblib 3,439
2 PSpider 1,785
3 aiomultiprocess 1,601
4 imgp 967
5 Plutus 958
6 crypto-rl 718
7 pytorch-A3C 538
8 vermin 392
9 lithops 280
10 think-async 214
11 multitasking 183
12 lox 87
13 ruck 52
14 NotEnoughAV1Encodes-Qt 32
15 google_drive_ocr 23
16 geventmp 17
17 regta 12
18 gibbs 11
19 mnqueues 5
20 treequeues 4
21 RemoteShell 2
22 autowpscan 2
23 mpxl2csv 1
Collect and Analyze Billions of Data Points in Real Time
Manage all types of time series data in a single, purpose-built database. Run at any scale in any environment in the cloud, on-premises, or at the edge.