Top 10 Python Concurrency and Parallelism Projects
An open source framework that provides a simple, universal API for building distributed applications. Ray is packaged with RLlib, a scalable reinforcement learning library, and Tune, a scalable hyperparameter tuning library.Project mention: JORLDY: OpenSource Reinforcement Learning Framework | reddit.com/r/reinforcementlearning | 2021-11-08
Distributed RL algorithms are provided using ray
Python Stream ProcessingProject mention: Event-Driven Architectures with Kafka and Python | reddit.com/r/apachekafka | 2021-10-25
The best bet seemed like the open source Faust (from Robinhood, and Celery in its lineage). It was too heavy duty for our simple needs and moreover it unfortunately seems abandoned. There is a community fork in a bit of disarray last i looked(tests failed etc.)
Scout APM: A developer's best friend. Try free for 14-days. Scout APM uses tracing logic that ties bottlenecks to source code so you know the exact line of code causing performance issues and can get back to building a great product faster.
Coroutine-based concurrency library for PythonProject mention: I have a problem with installing Ajenti on a 64bit Ubuntu 21.04 server | reddit.com/r/webdev | 2021-06-08
Greenlet seems to have some troubles compiling with Python 3.9. https://github.com/gevent/gevent/issues/1627
Project mention: Making lunch faster with Python concurrency | reddit.com/r/Python | 2021-03-17
Magic decorator syntax for asynchronous code in Python
Concurrent networking library for Python
SCOOP (Scalable COncurrent Operations in Python) (by soravux)
Run Linux Software Faster and Safer than Linux with Unikernels.
Python Actor concurrency libraryProject mention: Ask HN: How to 100% enable remote infrastructure? | news.ycombinator.com | 2021-04-29
"Even things that can't go wrong, do." - Troubleshooting Analog Circuits, Robert "Bob" Pease.
I had an application on a Raspberry PI that paired with a Bluetooth Low Energy device to fetch and send its data through a 3G dongle, on the premises of non-technical people who cannot troubleshoot, in different countries and time zones. There were a lot of things that could go wrong, and I wrote code to mitigate and recover, including pulling new code.
Part of it was using the Actor Model. I wrote actors to connect to the device, pulling data, sending data, computing what was sent, etc. The actor system handled the actors, when one died, it would recreate another one when an unhandled exception was met, for example.
If you're doing Scala, take a look at Akka. The new Scala has native support for this, if I remember correctly, without Akka.
CSP-style concurrency for Python
Python Concurrency and Parallelism related posts
JORLDY: OpenSource Reinforcement Learning Framework
2 projects | reddit.com/r/reinforcementlearning | 8 Nov 2021
Event-Driven Architectures with Kafka and Python
2 projects | reddit.com/r/apachekafka | 25 Oct 2021
Python stands to lose its GIL, and gain a lot of speed
5 projects | reddit.com/r/programming | 20 Oct 2021
How to deploy a rllib-trained model?
3 projects | reddit.com/r/reinforcementlearning | 16 Oct 2021
[HELP] Converting many individual workstations into a HPC cluster
1 project | reddit.com/r/HPC | 11 Oct 2021
Show HN: SpotML – Managed ML Training on Cheap AWS/GCP Spot Instances
6 projects | news.ycombinator.com | 3 Oct 2021
Writing your First Distributed Python Application with Ray (without multiprocessing)
4 projects | reddit.com/r/Python | 23 Aug 2021
What are some of the best open-source Concurrency and Parallelism projects in Python? This list will help you:
|7||SCOOP (Scalable COncurrent Operations in Python)||518|
|8||Thespian Actor Library||157|
Are you hiring? Post a new remote job listing for free.