datasets
FedScale
Our great sponsors
| datasets | FedScale | |
|---|---|---|
| 5 | 4 | |
| 4,175 | 366 | |
| 1.5% | 3.3% | |
| 9.4 | 7.9 | |
| 4 days ago | 4 months ago | |
| Python | Python | |
| Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
datasets
-
TensorFlow Datasets (TFDS): a collection of ready-to-use datasets
I tried Librispeech, a very common dataset for speech recognition, in both HF and TFDS.
TFDS performed extremely bad.
First it failed because the official hosting server only allows 5 simultaneous connections, and TFDS totally ignored that and makes up to 50 simultaneous downloads and that breaks. I wonder if anyone actually tested this?
Then you need to have some computer with 30GB to do the preparation, which might fail on your computer. This is where I stopped. https://github.com/tensorflow/datasets/issues/3887. It might be fixed now but it took them 8 months to respond to my issue.
On HF, it just worked. There was a smaller issue in how the dataset was split up but that is fixed now, and their response was very fast and great.
-
We built a pi controlled hydroponics box that grows your plants 1.5x faster using ML
but it looks like none of your plants are supported by the plantvillage model, or do I understand something wrong? https://github.com/tensorflow/datasets/blob/master/tensorflow_datasets/image_classification/plant_village.py#L57
-
Voice Recognition with Tensorflow
To do our example, we're going to use some audio files released by Google.
FedScale
-
University of Michigan Researchers Open-Source ‘FedScale’: a Federated Learning (FL) Benchmarking Suite with Realistic Datasets and a Scalable Runtime to Enable Reproducible FL Research on Privacy-Preserving Machine Learning
Continue reading | Checkout the paper, github link
- We created the most comprehensive benchmark datasets for federated learning to date!
- The most comprehensive benchmark datasets for federated learning to date!
-
The most comprehensive benchmark datasets for federated learning to date
We created FedScale, which has a diverse set of challenging and realistic benchmark datasets to facilitate scalable, comprehensive, and reproducible federated learning (FL) research. FedScale datasets are large-scale, encompassing a diverse range of important FL tasks, such as image classification, object detection, word prediction, and speech recognition. Our evaluation platform provides flexible APIs to implement new FL algorithms and includes new execution backends with minimal developer efforts. Check it out, and feel free to join the FedScale community via Slack(https://join.slack.com/t/fedscale/shared_invite/zt-uzouv5wh-ON8ONCGIzwjXwMYDC2fiKw)!
Paper: https://arxiv.org/abs/2105.11367 and Github repo: https://github.com/symbioticlab/fedscale
Cheers!
What are some alternatives?
Activeloop Hub - Data Lake for Deep Learning. Build, manage, query, version, & visualize datasets. Stream data real-time to PyTorch/TensorFlow. https://activeloop.ai [Moved to: https://github.com/activeloopai/deeplake]
flower - Flower: A Friendly Federated Learning Framework
flax - Flax is a neural network library for JAX that is designed for flexibility.
FederatedScope - An easy-to-use federated learning platform
jax-models - Unofficial JAX implementations of deep learning research papers
fedjax - FedJAX is a JAX-based open source library for Federated Learning simulations that emphasizes ease-of-use in research.
trax - Trax — Deep Learning with Clear Code and Speed
ORBIT-Dataset - The ORBIT dataset is a collection of videos of objects in clean and cluttered scenes recorded by people who are blind/low-vision on a mobile phone. The dataset is presented with a teachable object recognition benchmark task which aims to drive few-shot learning on challenging real-world data.
jaxopt - Hardware accelerated, batchable and differentiable optimizers in JAX.
FATE - An Industrial Grade Federated Learning Framework
ESC-50 - ESC-50: Dataset for Environmental Sound Classification
breaching - Breaching privacy in federated learning scenarios for vision and text