alpa
FedML
| alpa | FedML | |
|---|---|---|
| 4 | 6 | |
| 2,986 | 4,062 | |
| 0.8% | 0.9% | |
| 5.1 | 9.9 | |
| 5 months ago | 4 days ago | |
| Python | Python | |
| Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
alpa
-
How to Train Large Models on Many GPUs?
- Alpa does training and serving with 175B parameter models https://github.com/alpa-projects/alpa
-
how much does it actually cost in terms of computer power for open AI to respond
alpa.ai states "You will need at least 350GB GPU memory on your entire cluster to serve the OPT-175B model. For example, you can use 4 x AWS p3.16xlarge instances, which provide 4 (instance) x 8 (GPU/instance) x 16 (GB/GPU) = 512 GB memory."
- Alpa: Auto-parallelizing large model training and inference (by UC Berkeley)
-
Alpa: Automated Model-Parallel Deep Learning
GitHub code: https://github.com/alpa-projects/alpa
FedML
-
[Experiment] The future of AI is open-source, and here is the plan
FedML https://github.com/FedML-AI/FedML might already provide a lot of tools to do the job
- Awesome-Federated-Learning: A curated list of federated learning publications, re-organized from Arxiv (mostly).
-
Launch HN: Flower (YC W23) – Train AI models on distributed or sensitive data
This is not new at all. There is a much stronger competitor existing in the market already: FedML (https://fedml.ai). They have a much larger open-source community, and a well-managed and widely-used MLOps (https://open.fedml.ai).
-
FedML has just released its completely revamped and redesigned AI Platform and Website.
Website: https://fedml.ai Platform: https://open.fedml.ai/
- FedML AI platform releases the world’s federated learning open platform on the public cloud with an in-depth introduction of products and technologies!
- [Discussion] How feasible is it to partition a DNN model into pieces?
What are some alternatives?
transformers - 🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
federated-xgboost - Federated gradient boosted decision tree learning
hivemind - Decentralized deep learning in PyTorch. Built to train models on thousands of volunteers across the world.
experta - Expert Systems for Python
determined - Determined is an open-source machine learning platform that simplifies distributed training, hyperparameter tuning, experiment tracking, and resource management. Works with PyTorch and TensorFlow.
HandyRL - HandyRL is a handy and simple framework based on Python and PyTorch for distributed reinforcement learning that is applicable to your own environments.
awesome-tensor-compilers - A list of awesome compiler projects and papers for tensor computation and deep learning.
adaptdl - Resource-adaptive cluster scheduler for deep learning training.
MetisFL - The first open Federated Learning framework implemented in C++ and Python.
PaddlePaddle - PArallel Distributed Deep LEarning: Machine Learning Framework from Industrial Practice （『飞桨』核心框架，深度学习&机器学习高性能单机、分布式训练和跨平台部署）