SGD with Adaptive Preconditioning: Unified Analysis and Momentum Acceleration

Dmitry Kovalev

In this paper, we revisit stochastic gradient descent (SGD) with AdaGrad-type preconditioning. Our contributions are twofold. First, we develop a unified convergence analysis of SG…