Meta-Regularization: An Approach to Adaptive Choice of the Learning Rate in Gradient Descent

Guangzeng Xie, Hao Jin, Dachao Lin +1

We propose \textit{Meta-Regularization}, a novel approach for the adaptive choice of the learning rate in first-order gradient descent methods. Our approach modifies the objective…