The Impact of Anisotropic Covariance Structure on the Training Dynamics and Generalization Error of Linear Networks

Taishi Watanabe, Ryo Karakida, Jun-nosuke Teramae

The success of deep neural networks largely depends on the statistical structure of the training data. While learning dynamics and generalization on isotropic data are well-establi…