Scaling Limits of Constant-Stepsize SGD at Flat Minima

Jingyi Zhang, Cheng Mao, Debankur Mukherjee

For stochastic gradient descent (SGD) with a constant stepsize $\hat{I}\pm$, the invariant law of the iterates, centered at a minimizer, describes the behavior of the algorithm over long t…