Statistical Mechanics of Deep Linear Neural Networks: The Back-Propagating Kernel Renormalization

Qianyi Li, Haim Sompolinsky

The success of deep learning in many real-world tasks has triggered an intense effort to understand the power and limitations of deep learning in the training and generalization of…