Implicit Regularization of Stochastic Gradient Descent in Natural Language Processing: Observations and Implications

Deren Lei, Zichen Sun, Yijun Xiao +1

Deep neural networks with remarkably strong generalization performances are usually over-parameterized. Despite explicit regularization strategies are used for practitioners to avo…