Python model-parallelism Projects

ColossalAI

28 29,867 9.3 Python

Making large AI models cheaper, faster and more accessible

Project mention: ColossalChat: An Open-Source Solution for Cloning ChatGPT with a RLHF Pipeline | news.ycombinator.com | 2023-04-04

> open-source a complete RLHF pipeline ... based on the LLaMA pre-trained model
I've gotten to where when I see "open source AI" I now know it's "well, except for $some_other_dependencies"
Anyway: https://scribe.rip/@yangyou_berkeley/colossalchat-an-open-so... and https://github.com/hpcaitech/ColossalAI#readme (Apache 2) can save you some medium.com heartache at least
DeepSpeed

27 25,088 9.6 Python

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

Project mention: Using --deepspeed requires lots of manual tweaking | reddit.com/r/Oobabooga | 2023-05-11

Filed a discussion item on the deepspeed project: https://github.com/microsoft/DeepSpeed/discussions/3531
Sonar

www.sonarsource.com
sponsored

Write Clean Python Code. Always.. Sonar helps you commit clean code every time. With over 225 unique rules to find Python bugs, code smells & vulnerabilities, Sonar finds the issues while you focus on the work.

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2023-05-11.

Index

	Project	Stars
1	ColossalAI	29,867
2	DeepSpeed	25,088

Python model-parallelism

Python model-parallelism Projects

ColossalAI

DeepSpeed

Sonar

Index