Stochastic gradient descent introduces an effective landscape-dependent regularization favoring flat solutions

Ning Yang, Chao Tang, Yuhai Tu

Generalization is one of the most important problems in deep learning (DL). In the overparameterized regime in neural networks, there exist many low-loss solutions that fit the tra…