DeepSpeed
ColossalAI
Our great sponsors
- InfluxDB - Access the most powerful time series database as a service
- Sonar - Write Clean Python Code. Always.
- CodiumAI - TestGPT | Generating meaningful tests for busy devs
- ONLYOFFICE ONLYOFFICE Docs — document collaboration in your environment
| DeepSpeed | ColossalAI | |
|---|---|---|
| 41 | 40 | |
| 25,088 | 29,867 | |
| 61.0% | 34.9% | |
| 9.6 | 9.3 | |
| 2 days ago | about 16 hours ago | |
| Python | Python | |
| Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
DeepSpeed
-
Using --deepspeed requires lots of manual tweaking
Filed a discussion item on the deepspeed project: https://github.com/microsoft/DeepSpeed/discussions/3531
Solution: I don't know; this is where I am stuck. https://github.com/microsoft/DeepSpeed/issues/1037 suggests that I just need to 'apt install libaio-dev', but I've done that and it doesn't help.
-
Whether the ML computation engineering expertise will be valuable, is the question.
There could be some spectrum of this expertise. For instance, https://github.com/NVIDIA/FasterTransformer, https://github.com/microsoft/DeepSpeed
- FLiPN-FLaNK Stack Weekly for 17 April 2023
- DeepSpeed Chat: Easy, Fast and Affordable RLHF Training of ChatGPT-Like Models
- DeepSpeed-Chat: Easy, Fast and Affordable RLHF Training of ChatGPT-Like Models
-
12-Apr-2023 AI Summary
DeepSpeed Chat: Easy, Fast and Affordable RLHF Training of ChatGPT-like Models at All Scales (https://github.com/microsoft/DeepSpeed/tree/master/blogs/deepspeed-chat)
- Microsoft DeepSpeed
-
Apple: Transformer architecture optimized for Apple Silicon
I'm following this closely, together with other efforts like GPTQ Quantization and Microsoft's DeepSpeed, all of which are bringing down the hardware requirements of these advanced AI models.
-
Facebook LLAMA is being openly distributed via torrents
- https://github.com/microsoft/DeepSpeed
Anything that could bring this to a 10GB 3080 or 24GB 3090 without 60s/it per token?
ColossalAI
-
Open source solution replicates ChatGPT training process
The article talks about their RLHF implementation briefly. There’s details on their RLHF implementation here: https://github.com/hpcaitech/ColossalAI/blob/a619a190df71ea3...
-
An Open-Source Version of ChatGPT is Coming [News]
Need to deploy the inference model with Colossal AI.
-
Training dreambooth/embeddings on an RTX 3060 - possible?
It’s a framework for alot of pipeline parallelism optimizations that can allow you to not have to fit the whole model in vram. https://www.hpc-ai.tech/blog/diffusion-pretraining-and-hardware-fine-tuning-can-be-almost-7x-cheaper Tutorial here: https://github.com/hpcaitech/ColossalAI/blob/main/examples/images/dreambooth/README.md I have amd cards so I haven’t tried this yet but am thinking of converting my amd gpu server over to nvidia bc of this
-
A complete open-source solution for accelerating Stable Diffusion
Hey forks. We just release a complete open-source solution for accelerating Stable Diffusion pretraining and fine-tuning. It help reduce the pretraining cost by 6.5 times, and the hardware cost of fine-tuning by 7 times, while simultaneously speeding up the processes.
Open source address: https://github.com/hpcaitech/ColossalAI/tree/main/examples/images/diffusion
Our codebase for the diffusion models builds heavily on OpenAI's ADM codebase , lucidrains, Stable Diffusion, Lightning and Hugging Face. Thanks for open-sourcing!
We also write a blog post about it. https://medium.com/@yangyou_berkeley/diffusion-pretraining-and-hardware-fine-tuning-can-be-almost-7x-cheaper-85e970fe207b
Glad to know your thoughts about our work!
Just to make the links clickable:
https://github.com/hpcaitech/ColossalAI/tree/main/examples/i...
https://medium.com/@yangyou_berkeley/diffusion-pretraining-a...
-
We just release a complete open-source solution for accelerating Stable Diffusion pretraining and fine-tuning!
Open source address: https://github.com/hpcaitech/ColossalAI/tree/main/examples/images/diffusion
- Colossal-AI releases a complete open-source Stable Diffusion pretraining and fine-tuning solution that reduces the pretraining cost by 6.5 times, and the hardware cost of fine-tuning by 7 times, while simultaneously speeding up the processes
-
Colossal-AI Seamlessly Accelerates Large Models at Low Costs with Hugging Face
Portal Project address: https://github.com/hpcaitech/ColossalAI Reference https://arxiv.org/abs/2202.05924v2 https://arxiv.org/abs/2205.11487 https://github.com/features/copilot https://github.com/huggingface/transformers https://www.forbes.com/sites/forbestechcouncil/2022/03/25/six-ai-trends-to-watch-in-2022/?sh=4dc51f82be15 https://www.infoq.com/news/2022/06/meta-opt-175b/
-
The 10 Trending Python Repositories on GitHub (May 2022)
ColossalAI
What are some alternatives?
fairscale - PyTorch extensions for high performance and large scale training.
TensorRT - NVIDIA® TensorRT™, an SDK for high-performance deep learning inference, includes a deep learning inference optimizer and runtime that delivers low latency and high throughput for inference applications.
Megatron-LM - Ongoing research training transformer models at scale
fairseq - Facebook AI Research Sequence-to-Sequence Toolkit written in Python.
mesh-transformer-jax - Model parallel transformers in JAX and Haiku
llama - Inference code for LLaMA models
gpt-neox - An implementation of model parallel autoregressive transformers on GPUs, based on the DeepSpeed library.
server - The Triton Inference Server provides an optimized cloud and edge inferencing solution.
text-generation-webui - A gradio web UI for running Large Language Models like LLaMA, llama.cpp, GPT-J, Pythia, OPT, and GALACTICA.
determined - Determined: Deep Learning Training Platform
Finetune_LLMs - Repo for fine-tuning GPTJ and other GPT models