Overparameterized ReLU Neural Networks Learn the Simplest Models: Neural Isometry and Exact Recovery

Yifei Wang, Yixuan Hua, Emmanuel Candés +1

The practice of deep learning has shown that neural networks generalize remarkably well even with an extreme number of learned parameters. This appears to contradict traditional st…