Interpreting Adaptive Gradient Methods by Parameter Scaling for Learning-Rate-Free Optimization

Min-Kook Suh, Seung-Woo Seo

We address the challenge of estimating the learning rate for adaptive gradient methods used in training deep neural networks. While several learning-rate-free approaches have been…