FedScale
datasets
Our great sponsors
| FedScale | datasets | |
|---|---|---|
| 4 | 5 | |
| 363 | 4,157 | |
| 2.5% | 1.0% | |
| 7.9 | 9.3 | |
| 4 months ago | 6 days ago | |
| Python | Python | |
| Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
FedScale
-
University of Michigan Researchers Open-Source ‘FedScale’: a Federated Learning (FL) Benchmarking Suite with Realistic Datasets and a Scalable Runtime to Enable Reproducible FL Research on Privacy-Preserving Machine Learning
Continue reading | Checkout the paper, github link
- We created the most comprehensive benchmark datasets for federated learning to date!
- The most comprehensive benchmark datasets for federated learning to date!
-
The most comprehensive benchmark datasets for federated learning to date
We created FedScale, which has a diverse set of challenging and realistic benchmark datasets to facilitate scalable, comprehensive, and reproducible federated learning (FL) research. FedScale datasets are large-scale, encompassing a diverse range of important FL tasks, such as image classification, object detection, word prediction, and speech recognition. Our evaluation platform provides flexible APIs to implement new FL algorithms and includes new execution backends with minimal developer efforts. Check it out, and feel free to join the FedScale community via Slack(https://join.slack.com/t/fedscale/shared_invite/zt-uzouv5wh-ON8ONCGIzwjXwMYDC2fiKw)!
Paper: https://arxiv.org/abs/2105.11367 and Github repo: https://github.com/symbioticlab/fedscale
Cheers!
datasets
-
TensorFlow Datasets (TFDS): a collection of ready-to-use datasets
I tried Librispeech, a very common dataset for speech recognition, in both HF and TFDS.
TFDS performed extremely bad.
First it failed because the official hosting server only allows 5 simultaneous connections, and TFDS totally ignored that and makes up to 50 simultaneous downloads and that breaks. I wonder if anyone actually tested this?
Then you need to have some computer with 30GB to do the preparation, which might fail on your computer. This is where I stopped. https://github.com/tensorflow/datasets/issues/3887. It might be fixed now but it took them 8 months to respond to my issue.
On HF, it just worked. There was a smaller issue in how the dataset was split up but that is fixed now, and their response was very fast and great.
-
We built a pi controlled hydroponics box that grows your plants 1.5x faster using ML
but it looks like none of your plants are supported by the plantvillage model, or do I understand something wrong? https://github.com/tensorflow/datasets/blob/master/tensorflow_datasets/image_classification/plant_village.py#L57
-
Voice Recognition with Tensorflow
To do our example, we're going to use some audio files released by Google.
What are some alternatives?
flower - Flower: A Friendly Federated Learning Framework
Activeloop Hub - Data Lake for Deep Learning. Build, manage, query, version, & visualize datasets. Stream data real-time to PyTorch/TensorFlow. https://activeloop.ai [Moved to: https://github.com/activeloopai/deeplake]
FederatedScope - An easy-to-use federated learning platform
flax - Flax is a neural network library for JAX that is designed for flexibility.
fedjax - FedJAX is a JAX-based open source library for Federated Learning simulations that emphasizes ease-of-use in research.
jax-models - Unofficial JAX implementations of deep learning research papers
ORBIT-Dataset - The ORBIT dataset is a collection of videos of objects in clean and cluttered scenes recorded by people who are blind/low-vision on a mobile phone. The dataset is presented with a teachable object recognition benchmark task which aims to drive few-shot learning on challenging real-world data.
jaxopt - Hardware accelerated, batchable and differentiable optimizers in JAX.
FATE - An Industrial Grade Federated Learning Framework
einops - Flexible and powerful tensor operations for readable and reliable code (for pytorch, jax, TF and others)
automlbenchmark - OpenML AutoML Benchmarking Framework
trax - Trax — Deep Learning with Clear Code and Speed