ColossalAI
ivy
Our great sponsors
ColossalAI | ivy | |
---|---|---|
41 | 17 | |
37,465 | 13,980 | |
3.2% | 0.5% | |
9.7 | 10.0 | |
5 days ago | 6 days ago | |
Python | Python | |
Apache License 2.0 | GNU General Public License v3.0 or later |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
ColossalAI
-
Open source solution replicates ChatGPT training process
The article talks about their RLHF implementation briefly. There’s details on their RLHF implementation here: https://github.com/hpcaitech/ColossalAI/blob/a619a190df71ea3...
-
An Open-Source Version of ChatGPT is Coming [News]
Need to deploy the inference model with Colossal AI.
-
Training dreambooth/embeddings on an RTX 3060 - possible?
It’s a framework for alot of pipeline parallelism optimizations that can allow you to not have to fit the whole model in vram. https://www.hpc-ai.tech/blog/diffusion-pretraining-and-hardware-fine-tuning-can-be-almost-7x-cheaper Tutorial here: https://github.com/hpcaitech/ColossalAI/blob/main/examples/images/dreambooth/README.md I have amd cards so I haven’t tried this yet but am thinking of converting my amd gpu server over to nvidia bc of this
-
A complete open-source solution for accelerating Stable Diffusion
Hey forks. We just release a complete open-source solution for accelerating Stable Diffusion pretraining and fine-tuning. It help reduce the pretraining cost by 6.5 times, and the hardware cost of fine-tuning by 7 times, while simultaneously speeding up the processes.
Open source address: https://github.com/hpcaitech/ColossalAI/tree/main/examples/images/diffusion
Our codebase for the diffusion models builds heavily on OpenAI's ADM codebase , lucidrains, Stable Diffusion, Lightning and Hugging Face. Thanks for open-sourcing!
We also write a blog post about it. https://medium.com/@yangyou_berkeley/diffusion-pretraining-and-hardware-fine-tuning-can-be-almost-7x-cheaper-85e970fe207b
Glad to know your thoughts about our work!
Just to make the links clickable:
https://github.com/hpcaitech/ColossalAI/tree/main/examples/i...
https://medium.com/@yangyou_berkeley/diffusion-pretraining-a...
-
We just release a complete open-source solution for accelerating Stable Diffusion pretraining and fine-tuning!
Open source address: https://github.com/hpcaitech/ColossalAI/tree/main/examples/images/diffusion
- Colossal-AI releases a complete open-source Stable Diffusion pretraining and fine-tuning solution that reduces the pretraining cost by 6.5 times, and the hardware cost of fine-tuning by 7 times, while simultaneously speeding up the processes
-
Colossal-AI Seamlessly Accelerates Large Models at Low Costs with Hugging Face
Portal Project address: https://github.com/hpcaitech/ColossalAI Reference https://arxiv.org/abs/2202.05924v2 https://arxiv.org/abs/2205.11487 https://github.com/features/copilot https://github.com/huggingface/transformers https://www.forbes.com/sites/forbestechcouncil/2022/03/25/six-ai-trends-to-watch-in-2022/?sh=4dc51f82be15 https://www.infoq.com/news/2022/06/meta-opt-175b/
-
The 10 Trending Python Repositories on GitHub (May 2022)
ColossalAI
ivy
-
Keras 3.0
See also https://github.com/unifyai/ivy which I have not tried but seems along the lines of what you are describing, working with all the major frameworks
-
Show HN: Carton – Run any ML model from any programming language
is this ancillary to what [these guys](https://github.com/unifyai/ivy) are trying to do?
-
[D] Keras 3.0 Announcement: Keras for TensorFlow, JAX, and PyTorch
https://unify.ai/ They are trying to do what Ivy is doing already.
-
CoreML Stable Diffusion
ROCm's great for data centers, but good luck finding anything about desktop GPUs on their site apart from this lone blog post: https://community.amd.com/t5/instinct-accelerators/exploring...
There's a good explanation of AMD's ROCm targets here: https://news.ycombinator.com/item?id=28200477
It's currently a PITA to get common Python libs like Numba to even talk to AMD cards (admittedly Numba won't talk to older Nvidia cards either and they deprecate ruthlessly; I had to downgrade 8 versions to get it working with a 5yo mobile workstation). YC-backed Ivy claims to be working on unifying ML frameworks in a hardware-agnostic way but I don't have enough experience to assess how well they're succeeding yet: https://lets-unify.ai
I was happy to see DiffusionBee does talk the GPU in my late-model intel Mac, though for some reason it only uses 50% of its power right now. I'm sure the situation will improve as Metal 3.0 and Vulkan get more established.
-
[Discussion] Opinions on unify AI
What do you think about unify AI https://lets-unify.ai.
-
The coolest Python projects you've ever seen?
Ivy is seeking to unify all ML frameworks: https://lets-unify.ai/
-
The 10 Trending Python Repositories on GitHub (May 2022)
IVY
-
[P] Kornia: Differential Computer Vision
(*Differentiable) It's a great project. Wish they had a JAX version! Maybe something like Ivy would help make that possible without a manual port.
What are some alternatives?
DeepSpeed - DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
Megatron-LM - Ongoing research training transformer models at scale
determined - Determined is an open-source machine learning platform that simplifies distributed training, hyperparameter tuning, experiment tracking, and resource management. Works with PyTorch and TensorFlow.
fairscale - PyTorch extensions for high performance and large scale training.
DeepFaceLive - Real-time face swap for PC streaming or video calls
PaddleNLP - 👑 Easy-to-use and powerful NLP and LLM library with 🤗 Awesome model zoo, supporting wide-range of NLP tasks from research to industrial applications, including 🗂Text Classification, 🔍 Neural Search, ❓ Question Answering, ℹ️ Information Extraction, 📄 Document Intelligence, 💌 Sentiment Analysis etc.
PaddlePaddle - PArallel Distributed Deep LEarning: Machine Learning Framework from Industrial Practice (『飞桨』核心框架,深度学习&机器学习高性能单机、分布式训练和跨平台部署)
DeepFaceLab - DeepFaceLab is the leading software for creating deepfakes.
PaddleOCR - Awesome multilingual OCR toolkits based on PaddlePaddle (practical ultra lightweight OCR system, support 80+ languages recognition, provide data annotation and synthesis tools, support training and deployment among server, mobile, embedded and IoT devices)
lisp - Toy Lisp 1.5 interpreter