Using both Learning Rate Warmup and Learning Rate Decay Schedule in PyTorch

• pytorch_warmup

  3 360 3.4 Python

  Learning Rate Warmup in PyTorch

I’m currently using this for learning rate warmup, specifically the LinearWarmup(). So this simply ramps up from 0 to max_lr over a given number of steps.

• iamusica_training

  1 21 3.3 Python

  ONSETS&VELOCITIES real-time piano detection - PyTorch training [EUSIPCO2023]

Here is an implementation i made to allow for that kind of flexibility, hope it helps! https://github.com/andres-fr/iamusica_training/blob/master/ov_piano/optimizers.py

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

Suggest a related project