On the Activation Function Dependence of the Spectral Bias of Neural Networks

Qingguo Hong, Jonathan W. Siegel, Qinyang Tan +1

Neural networks are universal function approximators which are known to generalize well despite being dramatically overparameterized. We study this phenomenon from the point of vie…