Do Neural Networks Need Gradient Descent to Generalize? A Theoretical Study

Yotam Alexander, Yonatan Slutzky, Yuval Ran-Milo +1

Conventional wisdom attributes the mysterious generalization abilities of overparameterized neural networks to gradient descent (and its variants). The recent volume hypothesis cha…