47 citations · 47 across the 8 of their papers we have counts for
5 papers · 1 filter
A prism hierarchy of learning regimes in large linear autoencoders
Eugene Golikov, Yaroslav Gusev, Dmitry Yarotsky
Theoretical studies of machine learning models commonly consider different limiting regimes in which the learning dynamics of gradient descent becomes theoretically tractable. It i…
Gradient Flow Through Diagram Expansions: Learning Regimes and Explicit Solutions
Dmitry Yarotsky, Eugene Golikov, Yaroslav Gusev
We develop a general mathematical framework to analyze scaling regimes and derive explicit analytic solutions for gradient flow (GF) in large learning problems. Our key innovation…
SGD with memory: fundamental properties and stochastic acceleration
Dmitry Yarotsky, Maksim Velikanov
An important open problem is the theoretically feasible acceleration of mini-batch SGD-type algorithms on quadratic problems with power-law spectrum. In the non-stochastic setting,…
Generalization error of spectral algorithms
Maksim Velikanov, Maxim Panov, Dmitry Yarotsky
The asymptotically precise estimation of the generalization of kernel methods has recently received attention due to the parallels between neural networks and their associated kern…
Structure of universal formulas
Dmitry Yarotsky
By universal formulas we understand parameterized analytic expressions that have a fixed complexity, but nevertheless can approximate any continuous function on a compact set. Ther…