Learning Rate Perturbation: A Generic Plugin of Learning Rate Schedule towards Flatter Local Minima

Hengyu Liu, Qiang Fu, Lun Du +4

Learning rate is one of the most important hyper-parameters that has a significant influence on neural network training. Learning rate schedules are widely used in real practice to…