Gradient Methods with Online Scaling

Wenzhi Gao, Ya-Chi Chu, Yinyu Ye +1

We introduce a framework to accelerate the convergence of gradient-based methods with online learning. The framework learns to scale the gradient at each iteration through an onlin…