ffcv
ffcv-imagenet
| ffcv | ffcv-imagenet | |
|---|---|---|
| 8 | 1 | |
| 2,742 | 127 | |
| 0.7% | 2.4% | |
| 3.5 | 4.2 | |
| 8 days ago | 5 months ago | |
| Python | Python | |
| Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
ffcv
- Question: TIFF image dataset - size in RAM.
-
[P] Composer: a new PyTorch library to train models ~2-4x faster with better algorithms
PyTorch Lightning is also very slow compared to Composer. You don't have to believe us: our friends who wrote the FFCV library benchmarked us against PTL (see the lower left plot in the first cluster of graphs) , and you can see the difference for yourself. For the same accuracy, the FFCV folks found that Composer is about 5x faster than PTL on ResNet-50 on ImageNet.
- FFCV: Fast Forward Computer Vision
-
Does anyone know where I can find research papers for preprocessing large image datasets?
maybe something like this? https://github.com/libffcv/ffcv
- Ffcv: Train models at a fraction of the cost with accelerated data loading
- Show HN: FFCV – Accelerated machine learning via fast data loading
-
[P] FFCV: Accelerated Model Training via Fast Data Loading
Hi! You can join the slack directly from the link on the homepage! (ffcv.io)
ffcv-imagenet
-
[P] FFCV: Accelerated Model Training via Fast Data Loading
The best part about FFCV is how easy it is to use: chances are you can make your training code significantly faster by converting your dataset to FFCV format and changing just a few lines of code! To illustrate just how easy it is, we also made a minimal ImageNet example that gets high accuracies at SOTA speeds: https://github.com/libffcv/ffcv-imagenet.
What are some alternatives?
pytorch-lightning - Build high-performance AI models with PyTorch Lightning (organized PyTorch). Deploy models with Lightning Apps (organized Python to build end-to-end ML systems). [Moved to: https://github.com/Lightning-AI/lightning]
pillow-simd - The friendly PIL fork
best-of-ml-python - 🏆 A ranked list of awesome machine learning Python libraries. Updated weekly.
composer - Supercharge Your Model Training
apex - A PyTorch Extension: Tools for easy mixed precision and distributed training in Pytorch
array_storage_benchmark - Compare some methods of array storage in Python (numpy)
Pytorch - Tensors and Dynamic neural networks in Python with strong GPU acceleration
pytorch-lightning - Pretrain, finetune and deploy AI models on multiple GPUs, TPUs with zero code changes.
open_lth - A repository in preparation for open-sourcing lottery ticket hypothesis code.
webdataset - A high-performance Python-based I/O system for large (and small) deep learning problems, with strong support for PyTorch.