Lottery_Ticket_Hypothesis-TensorFlow_2 VS ydata-synthetic

I have already implemented "The Lottery Ticket Hypothesis" by Frankle et al. using TensorFlow 2. You can refer to the code here. Here, a binary mask (0, 1) is used for element-wise multiplication to keep the number of pruned parameters constant because by default, when you apply gradient descent algorithm, then using the weight update rule, all of the weights are updated.

Some of my recent experiments in GitHub can be referred: Lottery Ticket Hypothesis implementation and Neural Network Pruning.

I am using TensorFlow version: 2.3.0 and Python3. I am experimenting in Quantizing a pruned and trained Conv-2 CNN model. The model architecture is: conv -> conv -> max pool -> dense -> dense -> output for CIFAR-10. You can see the Jupyter-notebook here.

The code can be accessed here

here

True, removing he_normal initialization does increase the accuracy. For most of my previous experiments I have usually used the kernel initialization as mentioned in the different author's paper(s). Therefore for ResNet, I thought of using Kaiming He initialization as he is the author of the research paper. However, the default kernel initialization in TF2 is 'glorot_uniform' which leads to 60.04% val_accuracy.

Send us your North ⭐️: "On the first day of Christmas, my true contributor gave to me..." a star in my GitHub tree! 🎵 If you love these projects too, star ydata-profiling or ydata-synthetic and let your friends know why you love it so much!

1: Try to work with what you have and augment your dataset (honestly, 10 points is crap)

Data Quality is key for all applications and models, and LLMs are no exception :) I've been working on a small community project with synthetic data (https://github.com/ydataai/ydata-synthetic) using ydata-synthetic, and it really shows! Underrepresentation (category imbalance) and missing data are two of the main issues!

The Data-Centric AI Community creates community projects from time to time and is probably willing to help you in your project.

Join nice data communities and start networking.

You know what they say: "Tell me who your friends are, and I'll tell you who you are!". Hang out with DS beasts and learn from them :)

Interesting question! I think our AI/ML devs at the Data-Centric AI Community could have nice perspectives for your to decide :)

Sounds like an amazing journey! Feel free to add your projects on our awesome-python-for-data-science repo as you go! And in case you need a hand or feedback on the projects, we'll be happy to help at the Data-Centric AI Community.

The best to do is to get started with end-to-end projects in a collaborative environment (somewhat approaching real-world settings). You may find some interesting resources in this GitHub repository. The Data-Centric AI Community actually has a nice support system for this.