Automatic, Dynamic, and Nearly Optimal Learning Rate Specification by Local Quadratic Approximation

Yingqiu Zhu, Yu Chen, Danyang Huang +2

In deep learning tasks, the learning rate determines the update step size in each iteration, which plays a critical role in gradient-based optimization. However, the determination…