Python Distributed Computing

Open-source Python projects categorized as Distributed Computing
Edit details
Machine Learning Python Deep Learning Distributed Systems AI
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com
featured

Top 22 Python Distributed Computing Projects

Distributed Computing

  1. ColossalAI

    Making large AI models cheaper, faster and more accessible

    Project mention: FLaNK AI-April 22, 2024 | dev.to | 2024-04-22

  2. SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo

  3. catalyst

    Accelerated deep learning R&D (by catalyst-team)

  4. rl

    A modular, primitive-first, python-first PyTorch library for Reinforcement Learning. (by pytorch)

  5. fugue

    A unified interface for distributed computing. Fugue executes SQL, Python, Pandas, and Polars code on Spark, Dask and Ray without any rewrites.

  6. distributed

    A distributed task scheduler for Dask

  7. vizier

    Python-based research interface for blackbox and hyperparameter optimization, based on the internal Google Vizier Service.

  8. AI-Horde

    A crowdsourced distributed cluster for AI art and text generation

  9. couler

    Unified Interface for Constructing and Managing Workflows on different workflow engines, such as Argo Workflows, Tekton Pipelines, and Apache Airflow.

  10. bagua

    Bagua Speeds up PyTorch

  11. tdigest

    t-Digest data structure in Python. Useful for percentiles and quantiles, including distributed enviroments like PySpark (by CamDavidsonPilon)

  12. machinaris

    An easy-to-use WebUI for crypto plotting and farming. Offers Bladebit, Gigahorse, MadMax, Chiadog and Plotman in a Docker container. Supports Chia, MMX, Chives, Flax, and HDDCoin among others.

  13. sparktorch

    Train and run Pytorch models on Apache Spark.

  14. arkouda

    Arkouda (αρκούδα): Interactive Data Analytics at Supercomputing Scale :bear:

  15. stable-diffusion-webui-distributed

    Chains stable-diffusion-webui instances together to facilitate faster image generation.

  16. mlToolKits

    learningOrchestra is a distributed Machine Learning integration tool that facilitates and streamlines iterative processes in a Data Science project.

  17. wrapyfi

    Python Multi-Middleware Wrapper for Robotics MOM and RPC

  18. redis-dict

    Python dictionary with Redis as backend, built for large datasets. Simplifies Redis operations for large-scale and distributed systems. Supports various data types, namespacing, pipelining, and expiration.

    Project mention: Show HN: RedisDict | news.ycombinator.com | 2024-11-05

    It handles types without Pickle since remote pickled data is unsafe. Built for working with large datasets, it implements the full dictionary interface with extensive test coverage.

    GitHub: https://github.com/Attumm/redis-dict

  19. tune

    An abstraction layer for parameter tuning (by fugue-project)

  20. FindTheMag2

    A tool to determine optimal projects for Gridcoin & BOINC crunchers. Maximize your magnitude!

  21. rxray

    Ray distributed computing integration for RxPY

  22. py-inventa

    A Python library for microservice registry and executing RPC (Remote Procedure Call) over Redis.

  23. hulse-py

    The Python client for the Hulse platform

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).

Python Distributed Computing discussion

Log in or Post with

Python Distributed Computing related posts

  • Show HN: RedisDict

    1 project | news.ycombinator.com | 5 Nov 2024

  • Show HN: Interactive Graph by LLM (GPT-4o)

    5 projects | news.ycombinator.com | 19 May 2024

  • Daft: A High-Performance Distributed Dataframe Library for Multimodal Data

    4 projects | news.ycombinator.com | 7 Jun 2023

  • about making a game.

    2 projects | /r/ArtificialInteligence | 6 Jun 2023

  • TIL : about the game "Foldit", a puzzle game about protein folding. In 2011, its gamers helped decipher a protein of a HIV-like virus, solving a scientific problem that went unsolved for 15 years in as little as 10 days.

    5 projects | /r/todayilearned | 22 May 2023

  • Alternatives to Kaggle and Collab?

    1 project | /r/StableDiffusion | 25 Apr 2023

  • Shuffling large data at constant memory in Dask

    1 project | /r/Python | 17 Apr 2023
  • A note from our sponsor - SaaSHub
    www.saashub.com | 24 Jan 2025
    SaaSHub helps you find the best software and product alternatives Learn more →

Index

What are some of the best open-source Distributed Computing projects in Python? This list will help you:

# Project Stars
1 ColossalAI 39,022
2 catalyst 3,316
3 rl 2,481
4 fugue 2,035
5 distributed 1,589
6 vizier 1,531
7 AI-Horde 1,127
8 couler 921
9 bagua 877
10 tdigest 382
11 machinaris 341
12 sparktorch 339
13 arkouda 256
14 stable-diffusion-webui-distributed 183
15 mlToolKits 75
16 wrapyfi 74
17 redis-dict 54
18 tune 36
19 FindTheMag2 28
20 rxray 12
21 py-inventa 8
22 hulse-py 6

Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com
Loading...