Spectral phase transitions and trainability in neural network learning dynamics

Chanju Park, Dario Bocchi, Francesco D'Amico +2

The emergence of low-dimensional structures in the spectra of neural network weight matrices is a common empirical feature of trained models, but the dynamical origin of this pheno…