DALI
mmap.ninja
| DALI | mmap.ninja | |
|---|---|---|
| 5 | 4 | |
| 4,917 | 264 | |
| 1.1% | - | |
| 9.6 | 7.5 | |
| 4 days ago | 3 months ago | |
| C++ | Python | |
| Apache License 2.0 | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
DALI
-
[D] Will data augmentations work faster on TPUs?
Another option is DALI https://github.com/NVIDIA/DALI For my project while training EfficientNet2, it was a game changer. But it a way harder to implement in code than TorchVision or Kornia.
-
DirectStorage - Loading data to GPU *directly* from the SSD drive, almost without using CPU
Check out https://github.com/nvidia/DALI
-
mmap_ninja: Speedup your training dramatically by using memory-mapped files for your dataset
Small question if you are using GPU: How to this compare to GPUDirect Storage from Nvidia? can you have even more speedup by using both? I never toy with it, but the DALI project from Nvidia seem to tackle the same data loading problem.
- [D] Efficiently loading videos in PyTorch without extracting frames
mmap.ninja
-
[P]: mmap_ninja: Speedup your training dramatically by using memory-mapped files for your dataset
Repo link: https://github.com/hristo-vrigazov/mmap.ninja
- mmap_ninja: Speedup your training dramatically by using memory-mapped files for your dataset
- Show HN: MMAP_ninja: a library fo storing ML datasets in memory mapped format
What are some alternatives?
Pytorch - Tensors and Dynamic neural networks in Python with strong GPU acceleration
Blurry - Blurry is an easy blur library for Android
vision - Datasets, Transforms and Models specific to Computer Vision
executorch - On-device AI across mobile, embedded and edge for PyTorch
DREAMPlace - Deep learning toolkit-enabled VLSI placement
MegEngine - MegEngine 是一个快速、可拓展、易于使用且支持自动求导的深度学习框架
ocaml-torch - OCaml bindings for PyTorch
SlowFast - PySlowFast: video understanding codebase from FAIR for reproducing state-of-the-art video models.
Image Steganography - ✔️ Hide a secret message in an image
soft_nms - PyTorch implementation of soft-nms
Louvre - A small customizable library useful to handle an gallery image pick action built-in your app. :sunrise_over_mountains::stars:
Backboard - A motion-driven animation framework for Android.