Regularization Matters: A Nonparametric Perspective on Overparametrized Neural Network

Tianyang Hu, Wenjia Wang, Cong Lin +1

Overparametrized neural networks trained by gradient descent (GD) can provably overfit any training data. However, the generalization guarantee may not hold for noisy data. From a…