Python Concurrency and Parallelism

Open-source Python projects categorized as Concurrency and Parallelism | Edit details

Top 10 Python Concurrency and Parallelism Projects

  • GitHub repo Ray

    An open source framework that provides a simple, universal API for building distributed applications. Ray is packaged with RLlib, a scalable reinforcement learning library, and Tune, a scalable hyperparameter tuning library.

    Project mention: JORLDY: OpenSource Reinforcement Learning Framework | reddit.com/r/reinforcementlearning | 2021-11-08

    Distributed RL algorithms are provided using ray

  • GitHub repo Faust

    Python Stream Processing

    Project mention: Event-Driven Architectures with Kafka and Python | reddit.com/r/apachekafka | 2021-10-25

    The best bet seemed like the open source Faust (from Robinhood, and Celery in its lineage). It was too heavy duty for our simple needs and moreover it unfortunately seems abandoned. There is a community fork in a bit of disarray last i looked(tests failed etc.)

  • Scout APM

    Scout APM: A developer's best friend. Try free for 14-days. Scout APM uses tracing logic that ties bottlenecks to source code so you know the exact line of code causing performance issues and can get back to building a great product faster.

  • GitHub repo gevent

    Coroutine-based concurrency library for Python

    Project mention: I have a problem with installing Ajenti on a 64bit Ubuntu 21.04 server | reddit.com/r/webdev | 2021-06-08

    Greenlet seems to have some troubles compiling with Python 3.9. https://github.com/gevent/gevent/issues/1627

  • GitHub repo deco

    Project mention: Making lunch faster with Python concurrency | reddit.com/r/Python | 2021-03-17
  • GitHub repo Tomorrow

    Magic decorator syntax for asynchronous code in Python

  • GitHub repo eventlet

    Concurrent networking library for Python

  • GitHub repo SCOOP (Scalable COncurrent Operations in Python)

    SCOOP (Scalable COncurrent Operations in Python) (by soravux)

  • Nanos

    Run Linux Software Faster and Safer than Linux with Unikernels.

  • GitHub repo Thespian Actor Library

    Python Actor concurrency library

    Project mention: Ask HN: How to 100% enable remote infrastructure? | news.ycombinator.com | 2021-04-29

    "Even things that can't go wrong, do." - Troubleshooting Analog Circuits, Robert "Bob" Pease.

    I had an application on a Raspberry PI that paired with a Bluetooth Low Energy device to fetch and send its data through a 3G dongle, on the premises of non-technical people who cannot troubleshoot, in different countries and time zones. There were a lot of things that could go wrong, and I wrote code to mitigate and recover, including pulling new code.

    Part of it was using the Actor Model. I wrote actors to connect to the device, pulling data, sending data, computing what was sent, etc. The actor system handled the actors, when one died, it would recreate another one when an unhandled exception was met, for example.

    https://en.wikipedia.org/wiki/Actor_model

    If you're doing Scala, take a look at Akka. The new Scala has native support for this, if I remember correctly, without Akka.

    If you're doing Python: https://github.com/kquick/Thespian, https://thespianpy.com/doc/

  • GitHub repo aiochan

    CSP-style concurrency for Python

  • GitHub repo pyeventbus

    Python Eventbus

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2021-11-08.

Python Concurrency and Parallelism related posts

Index

What are some of the best open-source Concurrency and Parallelism projects in Python? This list will help you:

Project Stars
1 Ray 18,352
2 Faust 5,884
3 gevent 5,649
4 deco 1,522
5 Tomorrow 1,461
6 eventlet 1,072
7 SCOOP (Scalable COncurrent Operations in Python) 518
8 Thespian Actor Library 157
9 aiochan 116
10 pyeventbus 15
Find remote jobs at our new job board 99remotejobs.com. There are 31 new remote jobs listed recently.
Are you hiring? Post a new remote job listing for free.
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com