Lottery_Ticket_Hypothesis-TensorFlow_2
labml
| Lottery_Ticket_Hypothesis-TensorFlow_2 | labml | |
|---|---|---|
| 6 | 23 | |
| 33 | 1,871 | |
| - | 2.1% | |
| 4.1 | 9.7 | |
| about 1 month ago | 2 days ago | |
| Jupyter Notebook | Jupyter Notebook | |
| - | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Lottery_Ticket_Hypothesis-TensorFlow_2
-
Freeze certain weights - TensorFlow 2
I have already implemented "The Lottery Ticket Hypothesis" by Frankle et al. using TensorFlow 2. You can refer to the code here. Here, a binary mask (0, 1) is used for element-wise multiplication to keep the number of pruned parameters constant because by default, when you apply gradient descent algorithm, then using the weight update rule, all of the weights are updated.
-
[R] Remove pruned connections
Some of my recent experiments in GitHub can be referred: Lottery Ticket Hypothesis implementation and Neural Network Pruning.
-
TensorFlow Lite: RuntimeError
I am using TensorFlow version: 2.3.0 and Python3. I am experimenting in Quantizing a pruned and trained Conv-2 CNN model. The model architecture is: conv -> conv -> max pool -> dense -> dense -> output for CIFAR-10. You can see the Jupyter-notebook here.
-
Iterative Pruning: LeNet-300-100 - PyTorch
The code can be accessed here
-
Neural Network Compression - Implementation benefits
here
-
ValueError: TensorFlow2 Input 0 is incompatible with layer model
True, removing he_normal initialization does increase the accuracy. For most of my previous experiments I have usually used the kernel initialization as mentioned in the different author's paper(s). Therefore for ResNet, I thought of using Kaiming He initialization as he is the author of the research paper. However, the default kernel initialization in TF2 is 'glorot_uniform' which leads to 60.04% val_accuracy.
labml
-
Creating stickers using SD with img2img
Used the PromptArt app by labml.ai to generate a sticker of an image I took from my iPhone. The results are amazing.
- [D] Why doesn’t your team use an experiment tracking tool?
-
Probe PyTorch models
💻 Github
-
[P] Probe PyTorch models
🧑🏫 Demo that extracts attention maps of BERT
- Show HN: Probe PyTorch Models
- [D] How do you guys tune hyperparameters, when a single training run takes a long time (days to weeks)?
- Machine Learning Best Practices
-
[D] Machine Learning Best Practices
from github
-
[P] Annotated deep learning paper implementations
labmlai/labml is a set of tools (tracking experiments, configurations, a bunch of helpers) we coded to ease our ML work (which later improved and open sourced). So we use it in all our projects because it makes things easier for us.
- React's UI State Model vs. Vanilla JavaScript
What are some alternatives?
Neural_Network_Pruning - Implementations of different neural network pruning techniques
nn - 🧑🏫 60 Implementations/tutorials of deep learning papers with side-by-side notes 📝; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, sophia, ...), gans(cyclegan, stylegan2, ...), 🎮 reinforcement learning (ppo, dqn), capsnet, distillation, ... 🧠
guildai - Experiment tracking, ML developer tools
Practical_RL - A course in reinforcement learning in the wild
Deep-Learning-Push-Up-Counter - Deep Learning approach to count the number of repetitions in a video of push ups or pull ups.
tensorflow-onnx - Convert TensorFlow, Keras, Tensorflow.js and Tflite models to ONNX
MIRNet-TFJS - TensorFlow JS models for MIRNet for low-light💡 image enhancement
spock - spock is a framework that helps manage complex parameter configurations during research and development of Python applications
tensorflow-deep-learning - All course materials for the Zero to Mastery Deep Learning with TensorFlow course.
YPDL-Build-a-movie-recommendation-engine-with-TensorFlow - In this tutorial, we are going to build a Restricted Boltzmann Machine using TensorFlow that will give us recommendations based on movies that have been watched already. The datasets we are going to use are acquired from GroupLens and contains movies, users, and movie ratings by these users.
mlops-course - Learn how to design, develop, deploy and iterate on production-grade ML applications.
text-to-text-transfer-transformer - Code for the paper "Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer"