Selecting and Composing Learning Rate Policies for Deep Neural Networks

Yanzhao Wu, Ling Liu

The choice of learning rate (LR) functions and policies has evolved from a simple fixed LR to the decaying LR and the cyclic LR, aiming to improve the accuracy and reduce the train…