Exponential convergence rates for momentum stochastic gradient descent in the overparametrized setting

Benjamin Gess, Sebastian Kassing

We prove explicit bounds on the exponential rate of convergence for the momentum stochastic gradient descent scheme (MSGD) for arbitrary, fixed hyperparameters (learning rate, fric…