NYU-DLSP20
webdataset
| NYU-DLSP20 | webdataset | |
|---|---|---|
| 2 | 7 | |
| 6,627 | 1,981 | |
| - | 4.2% | |
| 6.1 | 8.8 | |
| 3 months ago | 26 days ago | |
| Jupyter Notebook | Python | |
| GNU General Public License v3.0 or later | BSD 3-clause "New" or "Revised" License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
NYU-DLSP20
-
A collection of some of the best PyTorch courses for beginners to learn PyTorch online
And of course our NYU DL course 😉 https://github.com/Atcold/pytorch-Deep-Learning
-
Week 6 practicum notebook
I am going through week 6 practicum notebook. Can someone shed some light on the following code in train method:
webdataset
-
How to use data stored in a (private) S3 Bucket for training?
As an alternative, I've looked into using WebDataset, but couldn't figure out how to access data that is stored in a private bucket.
- [D] Title: Best tools and frameworks for working with million-billion image datasets?
-
[D] Training networks on extremely large datasets (10+TB)?
You can try webdataset (https://github.com/webdataset/webdataset).
- Question: TIFF image dataset - size in RAM.
-
How to upload large amounts of data to a server?
compress it to .tar format and then load it as a webdataset
-
Does mit 6.824 help for distributed deep learning?
Would guess not but there should be some good niche resources: check out the introductory videos here https://github.com/webdataset/webdataset
-
How to effectively load a large text dataset with PyTorch?
I found a pretty good solution that is similar to the TFRecord from Tensorflow. You just need to load the data, tokenized it, and save the arrays in shards with webdataset package.
What are some alternatives?
Real-time-Object-Detection-for-Autonomous-Driving-using-Deep-Learning - My Computer Vision project from my Computer Vision Course (Fall 2020) at Goethe University Frankfurt, Germany. Performance comparison between state-of-the-art Object Detection algorithms YOLO and Faster R-CNN based on the Berkeley DeepDrive (BDD100K) Dataset.
Practical_RL - A course in reinforcement learning in the wild
nlp-class - A Natural Language Processing course taught by Professor Ghassemi
Made-With-ML - Learn how to design, develop, deploy and iterate on production-grade ML applications.
bitcoin_price_prediction - This project tries to prediction the bitcoin price with machine and deep learning.
ffcv - FFCV: Fast Forward Computer Vision (and other ML workloads!)
ydata-profiling - 1 Line of code data quality profiling & exploratory data analysis for Pandas and Spark DataFrames.
fastai - The fastai deep learning library
dl-colab-notebooks - Try out deep learning models online on Google Colab
ModelNet40-C - Repo for "Benchmarking Robustness of 3D Point Cloud Recognition against Common Corruptions" https://arxiv.org/abs/2201.12296
ML-Workspace - 🛠All-in-one web-based IDE specialized for machine learning and data science.
PySyft - Perform data science on data that remains in someone else's server