Scaling Limits of Constant-Stepsize SGD at Flat Minima

Jingyi Zhang, Cheng Mao, Debankur Mukherjee

For stochastic gradient descent (SGD) with a constant stepsize $\alpha$, the invariant law of the iterates, centered at a minimizer, describes the behavior of the algorithm over long ti…