openfl
sparktorch
| openfl | sparktorch | |
|---|---|---|
| 4 | 1 | |
| 660 | 335 | |
| 1.7% | - | |
| 8.7 | 2.5 | |
| 6 days ago | 12 months ago | |
| Jupyter Notebook | Python | |
| Apache License 2.0 | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
openfl
-
Dectralized AI solutions/building blocks?
We have petals.ml for distributing running a single model large model for inference. We have [flower](flower.dev),tensor flow federated,openfl, etc for building federated learning.
- Launch HN: Flower (YC W23) – Train AI models on distributed or sensitive data
-
[R][P] OpenFL: An open-source framework for Federated Learning
Code for https://arxiv.org/abs/2105.06413 found: https://github.com/intel/openfl
sparktorch
-
Spark2 + pytorch on GPU
Was reading the documentation of sparktorch (https://github.com/dmmiller612/sparktorch) which says you need spark >= 2.4.4. But to the best of my knowledge spark2 doesn't have gpu compute capabilities. Does that mean it can only use cpu compute? Am I missing something?
What are some alternatives?
PySyft - Perform data science on data that remains in someone else's server
pytorch-lightning - Build high-performance AI models with PyTorch Lightning (organized PyTorch). Deploy models with Lightning Apps (organized Python to build end-to-end ML systems). [Moved to: https://github.com/Lightning-AI/lightning]
FATE - An Industrial Grade Federated Learning Framework
fastT5 - ⚡ boost inference speed of T5 models by 5x & reduce the model size by 3x.
fedjax - FedJAX is a JAX-based open source library for Federated Learning simulations that emphasizes ease-of-use in research.
torch2trt - An easy to use PyTorch to TensorRT converter
tf-encrypted - A Framework for Encrypted Machine Learning in TensorFlow
BERT-NER - Pytorch-Named-Entity-Recognition-with-BERT
FindTheMag - A tool to determine optimal projects for Gridcoin crunchers. Maximize your magnitude!
polyaxon - MLOps Tools For Managing & Orchestrating The Machine Learning LifeCycle
FedML - FEDML - The unified and scalable ML library for large-scale distributed training, model serving, and federated learning. FEDML Launch, a cross-cloud scheduler, further enables running any AI jobs on any GPU cloud or on-premise cluster. Built on this library, FEDML Nexus AI (https://fedml.ai) is your generative AI platform at scale.
pytorch-lightning - Pretrain, finetune and deploy AI models on multiple GPUs, TPUs with zero code changes.