Learning Rate: Decay, Warmup & Scheduler

• CNN_Classifications

  12 6 6.4 Jupyter Notebook

  Convolutional Neural Networks for image recognition and classification

While trying to implement different SOTA research papers, I had a roadblock in terms of finding working code for different learning rate: decay, warmup & schedules (piece-wise decay, step decay, exponential decay, etc.). Therefore, I created a Jupyter Notebook implementing these concepts using TensorFlow 2.4, Python3.8 specifically for custom training loops with tf.GradientTape since most of the tutorials/blogs only show "model.fit()" method. The code can be accessed here.

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

Suggest a related project