Towards Simple and Provable Parameter-Free Adaptive Gradient Methods

Yuanzhe Tao, Yifeng Liu, Huizhuo Yuan +3

Optimization algorithms such as AdaGrad and Adam have significantly advanced the training of deep models by dynamically adjusting the learning rate during the optimization process.…