Gradient Descent with Projection Finds Over-Parameterized Neural Networks for Learning Low-Degree Polynomials with Nearly Minimax Optimal Rate

Yingzhen Yang, Ping Li

We study the problem of learning a low-degree spherical polynomial of degree $k_0=\hat{I}˜(1)\ge 1$ defined on the unit sphere in $\RR^d$ by training an over-parameterized two-layer n…