Noise-Driven Exploration and Transient Freezing Select Flat Minima in Stochastic Gradient Descent

Ning Yang, Yikuan Zhang, Qi Ouyang +2

Stochastic gradient descent (SGD) is central to deep learning, yet the dynamical origin of its preference for flatter, more generalizable solutions remains unclear. Here, by analyz…