11 citations · 15 across the 5 of their papers we have counts for
4 papers · 1 filter
The Dynamics of Generalization in Deep Learning
Rubing Yang, Pratik Chaudhari
We derive a differential equation that governs the evolution of the generalization gap when a model is trained by gradient descent-based methods. This differential equation is driv…
The Training Process of Many Deep Networks Explores the Same Low-Dimensional Manifold
Jialin Mao, Itay Griniasty, Han Kheng Teoh +5
We develop information-geometric techniques to analyze the trajectories of the predictions of deep networks during training. By examining the underlying high-dimensional probabilis…
A picture of the space of typical learnable tasks
Rahul Ramesh, Jialin Mao, Itay Griniasty +5
We develop information geometric techniques to understand the representations learned by deep networks when they are trained on different tasks using supervised, meta-, semi-superv…
Does the Data Induce Capacity Control in Deep Learning?
Rubing Yang, Jialin Mao, Pratik Chaudhari
We show that the input correlation matrix of typical classification datasets has an eigenspectrum where, after a sharp initial drop, a large number of small eigenvalues are distrib…