3 citations · 3 across the 3 of their papers we have counts for
3 papers
Training Infinitely Deep and Wide Transformers
Raphaël Barboni, Maarten V. de Hoop, Takashi Furuya +1
Transformers have become the dominant architecture in modern machine learning, yet the theoretical understanding of their training dynamics remains limited. This paper develops a r…
Ultra-fast feature learning for the training of two-layer neural networks in the two-timescale regime
Raphaël Barboni, Gabriel Peyré, François-Xavier Vialard
We study the convergence of gradient methods for the training of mean-field single-hidden-layer neural networks with square loss. For this high-dimensional and non-convex optimizat…
Global convergence of ResNets: From finite to infinite width using linear parameterization
Raphaël Barboni, Gabriel Peyré, François-Xavier Vialard
Overparametrization is a key factor in the absence of convexity to explain global convergence of gradient descent (GD) for neural networks. Beside the well studied lazy regime, inf…