The Spectral Bias of Shallow Neural Network Learning is Shaped by the Choice of Non-linearity

Justin Sahs, Ryan Pyle, Fabio Anselmi +1

Despite classical statistical theory predicting severe overfitting, modern massively overparameterized neural networks still generalize well. This unexpected property is attributed…