2 citations · 5 across the 16 of their papers we have counts for
5 papers · 2 filters
Transformative or Conservative? Conservation laws for ResNets and Transformers
Sibylle Marcotte, Rémi Gribonval, Gabriel Peyré
While conservation laws in gradient flow training dynamics are well understood for (mostly shallow) ReLU and linear networks, their study remains largely unexplored for more practi…
Learning from samples: inverse problems over measures
Francisco Andrade, Gabriel Peyré, Clarice Poon
We study inverse problems where an unknown potential is observed only through samples from the measure it induces by a convex variational principle. Such problems arise in learning…
Ultra-fast feature learning for the training of two-layer neural networks in the two-timescale regime
Raphaël Barboni, Gabriel Peyré, François-Xavier Vialard
We study the convergence of gradient methods for the training of mean-field single-hidden-layer neural networks with square loss. For this high-dimensional and non-convex optimizat…
From Score Matching to Diffusion: A Fine-Grained Error Analysis in the Gaussian Setting
Samuel Hurault, Matthieu Terris, Thomas Moreau +1
Sampling from an unknown distribution, accessible only through discrete samples, is a fundamental problem at the core of generative AI. The current state-of-the-art methods follow…
A Unified Perspective on the Dynamics of Deep Transformers
Valérie Castin, Pierre Ablin, José Antonio Carrillo +1
Transformers, which are state-of-the-art in most machine learning tasks, represent the data as sequences of vectors called tokens. This representation is then exploited by the atte…