Top 12 Python Cluster Projects

TensorFlowOnSpark

2 3,863 1.4 Python

TensorFlowOnSpark brings TensorFlow programs to Apache Spark clusters.
dcos

1 2,339 0.0 Python

DC/OS - The Datacenter Operating System
InfluxDB

www.influxdata.com sponsored

Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
Udacity-Data-Engineering-Projects

5 1,295 0.0 Python

Few projects related to Data Engineering including Data Modeling, Infrastructure setup on cloud, Data Warehousing and Data Lake development.

Project mention: Pitanje za data engineering? | /r/programiranje | 2023-06-30

sparkmagic

4 1,284 7.6 Python

Jupyter magics and kernels for working with remote Spark clusters

Project mention: Doing ML works in AWS. Need help installing cartopy | /r/aws | 2023-06-05

Please file an issue at https://github.com/jupyter-incubator/sparkmagic

mlToolKits

1 74 0.0 Python

learningOrchestra is a distributed Machine Learning integration tool that facilitates and streamlines iterative processes in a Data Science project.
docker-stack-deploy

6 29 5.2 Python

Utility to improve docker stack deploy
sd-webui-discord-ex

1 25 8.3 Python

This is an extension of SD-WEBUI-DISCORD on the Stable Diffusion WebUI, which supports distributed deployment of SD node's Stable Diffusion WebUi Discord robots. The command usage on Discord can refer to the SD-WEBUI-DISCORD project.

Project mention: Stable Diffuision WebUI's discord bot extension: SD-WEBUI-DISCORD-EX | /r/StableDiffusion | 2023-08-26

WorkOS

workos.com sponsored

The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
ClusterMonitor

1 24 1.8 Python

Very simple script which logs the CPU and RAM usage of the submitted job.
docker-etcd-cluster

2 13 0.0 Python

Very simple etcd cluster powered by docker compose
woodKubernetes

1 4 0.0 Python

LXD wood cluster
backend

1 4 10.0 Python

The backend serving API, background tasks and DNS (by lxd-manager)
rpi-microk8s-bootstrap

1 3 2.6 Python

Automate RPI device conversion into Kubernetes cluster nodes with Terraform

Project mention: rpi-microk8s-bootstrap: Automate RPI device conversion into Kubernetes cluster nodes with Terraform | /r/microk8s | 2023-05-22

Github: https://github.com/AdnanHodzic/rpi-microk8s-bootstrap

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).

Python Cluster related posts

Are small ceph clusters viable?
3 projects | /r/ceph | 11 Jun 2023
Doing ML works in AWS. Need help installing cartopy
2 projects | /r/aws | 5 Jun 2023
Docker: Patroni + HAProxy + Etcd + PgBouncer
2 projects | /r/PostgreSQL | 28 Mar 2023
[D]Speed up inference on Spark
1 project | /r/MachineLearning | 18 Feb 2022
[D] Plug or Integrate a GNN Pytorch code base into Spark Cluster
2 projects | /r/MachineLearning | 3 Jan 2022
Very simple etcd cluster powered by docker compose
1 project | /r/docker | 8 Oct 2021
GitHub - Ne00n/woodKubernetes: LXD wood cluster
1 project | /r/LXD | 6 Aug 2021
A note from our sponsor - WorkOS
workos.com | 26 Apr 2024

The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning. Learn more →

Index

What are some of the best open-source Cluster projects in Python? This list will help you:

	Project	Stars
1	TensorFlowOnSpark	3,863
2	dcos	2,339
3	Udacity-Data-Engineering-Projects	1,295
4	sparkmagic	1,284
5	mlToolKits	74
6	docker-stack-deploy	29
7	sd-webui-discord-ex	25
8	ClusterMonitor	24
9	docker-etcd-cluster	13
10	woodKubernetes	4
11	backend	4
12	rpi-microk8s-bootstrap	3