An Effective Gram Matrix Characterizes Generalization in Deep Networks

Rubing Yang, Pratik Chaudhari

We derive a differential equation that governs the evolution of the generalization gap when a deep network is trained by gradient descent. This differential equation is controlled…