Does the Data Induce Capacity Control in Deep Learning?

Rubing Yang, Jialin Mao, Pratik Chaudhari

We show that the input correlation matrix of typical classification datasets has an eigenspectrum where, after a sharp initial drop, a large number of small eigenvalues are distrib…