Lottery_Ticket_Hypothesis-TensorFlow_2
Implementing "The Lottery Ticket Hypothesis" paper by "Jonathan Frankle, Michael Carbin" (by arjun-majumdar)
CNN_Classifications
Convolutional Neural Networks for image recognition and classification (by arjun-majumdar)
Lottery_Ticket_Hypothesis-TensorFlow_2 | CNN_Classifications | |
---|---|---|
6 | 12 | |
33 | 6 | |
- | - | |
4.1 | 6.4 | |
about 1 month ago | 4 months ago | |
Jupyter Notebook | Jupyter Notebook | |
- | - |
The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Lottery_Ticket_Hypothesis-TensorFlow_2
Posts with mentions or reviews of Lottery_Ticket_Hypothesis-TensorFlow_2.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2021-05-10.
-
Freeze certain weights - TensorFlow 2
I have already implemented "The Lottery Ticket Hypothesis" by Frankle et al. using TensorFlow 2. You can refer to the code here. Here, a binary mask (0, 1) is used for element-wise multiplication to keep the number of pruned parameters constant because by default, when you apply gradient descent algorithm, then using the weight update rule, all of the weights are updated.
-
[R] Remove pruned connections
Some of my recent experiments in GitHub can be referred: Lottery Ticket Hypothesis implementation and Neural Network Pruning.
-
TensorFlow Lite: RuntimeError
I am using TensorFlow version: 2.3.0 and Python3. I am experimenting in Quantizing a pruned and trained Conv-2 CNN model. The model architecture is: conv -> conv -> max pool -> dense -> dense -> output for CIFAR-10. You can see the Jupyter-notebook here.
-
Iterative Pruning: LeNet-300-100 - PyTorch
The code can be accessed here
-
Neural Network Compression - Implementation benefits
here
-
ValueError: TensorFlow2 Input 0 is incompatible with layer model
True, removing he_normal initialization does increase the accuracy. For most of my previous experiments I have usually used the kernel initialization as mentioned in the different author's paper(s). Therefore for ResNet, I thought of using Kaiming He initialization as he is the author of the research paper. However, the default kernel initialization in TF2 is 'glorot_uniform' which leads to 60.04% val_accuracy.
CNN_Classifications
Posts with mentions or reviews of CNN_Classifications.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2021-03-09.
-
Object Localization from scratch TF2
Object localization trained from scratch for emoji dataset in TensorFlow 2.8. Getting an IoU = 0.5969 and classification output accuracy = 100%. The code can be referred here. Though in fairness, I am using only 9 classes out of the emoji dataset. Thoughts?
-
ResNets PyTorch CIFAR-10
I have trained ResNet-18, ResNet-18 (dropout), ResNet-34 and ResNet-50 from scratch using He weights initializations and other SOTA practices and their implementations in Python 3.8 and PyTorch 1.8. ResNet-18/34 has a different architecture as compared to ResNet-50/101/152 due to bottleneck as specified by Kaiming He et al. in their research paper.
-
VGG-18 PyTorch
You can access the code here and here. According to some research papers, for deep learning architectures, using SGD vs. Adam optimizer leads to faster convergence.
-
PyTorch Pruning
I have implemented "Unstructured Global absolute magnitude" pruning using "torch.nn.utils.prune" with LeNet-5 trained on MNIST with iterative pruning. You can refer to the code here.
-
Learning Rate: Decay, Warmup & Scheduler
While trying to implement different SOTA research papers, I had a roadblock in terms of finding working code for different learning rate: decay, warmup & schedules (piece-wise decay, step decay, exponential decay, etc.). Therefore, I created a Jupyter Notebook implementing these concepts using TensorFlow 2.4, Python3.8 specifically for custom training loops with tf.GradientTape since most of the tutorials/blogs only show "model.fit()" method. The code can be accessed here.
-
ResNet from scratch - ImageNet
You can refer to my experiments here
-
ResNet-18 vs ResNet-34
Have a look at the code for manual early stopping using LeNet-300-100 FC Neural Network for MNIST dataset and let me know your thoughts.
I have trained ResNet-18 and ResNet-34 from scratch using PyTorch on CIFAR-10 dataset. The validation accuracy I get for ResNet-18 is 84.01%, whereas for ResNet-34 is 82.43%. Is this a sign of ResNet-34 overfitting as compared to ResNet-18? Ideally, ResNet-34 should achieve a higher validation accuracy as compared to ResNet-18.
-
ResNet-18 from scratch
I have implemented ResNet-18 CNN from scatch in Python and PyTorch using CIFAR-10 dataset. You can see it here.
-
ValueError: TensorFlow2 Input 0 is incompatible with layer model
I am trying to code a ResNet CNN architecture based on the paper by using Python3, TensorFlow2 and CIFAR-10 dataset. You can access the Jupyter notebook here.
What are some alternatives?
When comparing Lottery_Ticket_Hypothesis-TensorFlow_2 and CNN_Classifications you can also consider the following projects:
labml - 🔎 Monitor deep learning model training and hardware usage from your mobile phone 📱
Neural_Network_Pruning - Implementations of different neural network pruning techniques