CNN_Classifications VS pytorch-cifar

Object localization trained from scratch for emoji dataset in TensorFlow 2.8. Getting an IoU = 0.5969 and classification output accuracy = 100%. The code can be referred here. Though in fairness, I am using only 9 classes out of the emoji dataset. Thoughts?

I have trained ResNet-18, ResNet-18 (dropout), ResNet-34 and ResNet-50 from scratch using He weights initializations and other SOTA practices and their implementations in Python 3.8 and PyTorch 1.8. ResNet-18/34 has a different architecture as compared to ResNet-50/101/152 due to bottleneck as specified by Kaiming He et al. in their research paper.

You can access the code here and here. According to some research papers, for deep learning architectures, using SGD vs. Adam optimizer leads to faster convergence.

I have implemented "Unstructured Global absolute magnitude" pruning using "torch.nn.utils.prune" with LeNet-5 trained on MNIST with iterative pruning. You can refer to the code here.

While trying to implement different SOTA research papers, I had a roadblock in terms of finding working code for different learning rate: decay, warmup & schedules (piece-wise decay, step decay, exponential decay, etc.). Therefore, I created a Jupyter Notebook implementing these concepts using TensorFlow 2.4, Python3.8 specifically for custom training loops with tf.GradientTape since most of the tutorials/blogs only show "model.fit()" method. The code can be accessed here.

You can refer to my experiments here

Have a look at the code for manual early stopping using LeNet-300-100 FC Neural Network for MNIST dataset and let me know your thoughts.

I have trained ResNet-18 and ResNet-34 from scratch using PyTorch on CIFAR-10 dataset. The validation accuracy I get for ResNet-18 is 84.01%, whereas for ResNet-34 is 82.43%. Is this a sign of ResNet-34 overfitting as compared to ResNet-18? Ideally, ResNet-34 should achieve a higher validation accuracy as compared to ResNet-18.

I have implemented ResNet-18 CNN from scatch in Python and PyTorch using CIFAR-10 dataset. You can see it here.

I am trying to code a ResNet CNN architecture based on the paper by using Python3, TensorFlow2 and CIFAR-10 dataset. You can access the Jupyter notebook here.

I was playing with the CIFAR10 dataset based on the baseline code of https://github.com/kuangliu/pytorch-cifar, but I was surprised to see a strangely large decrease in the validation performance from using a smaller learning rate.

OP could also be underfitting. Some people have shown 93% accuracy on cifar10 using resnet18. See https://github.com/kuangliu/pytorch-cifar

This setup was originally based on the https://github.com/kuangliu/pytorch-cifar repo.