A prism hierarchy of learning regimes in large linear autoencoders

Eugene Golikov, Yaroslav Gusev, Dmitry Yarotsky

Theoretical studies of machine learning models commonly consider different limiting regimes in which the learning dynamics of gradient descent becomes theoretically tractable. It i…