Benign Oscillation of Stochastic Gradient Descent with Large Learning Rates

Miao Lu, Beining Wu, Xiaodong Yang +1

In this work, we theoretically investigate the generalization properties of neural networks (NN) trained by stochastic gradient descent (SGD) algorithm with large learning rates. U…