14 citations · 30 across the 11 of their papers we have counts for
4 papers · 1 filter
Global convergence of ResNets: From finite to infinite width using linear parameterization
Raphaël Barboni, Gabriel Peyré, François-Xavier Vialard
Overparametrization is a key factor in the absence of convexity to explain global convergence of gradient descent (GD) for neural networks. Beside the well studied lazy regime, inf…
Randomized Stochastic Gradient Descent Ascent
Othmane Sebbouh, Marco Cuturi, Gabriel Peyré
An increasing number of machine learning problems, such as robust or adversarial variants of existing algorithms, require minimizing a loss function that is itself defined as a max…
Sinkformers: Transformers with Doubly Stochastic Attention
Michael E. Sander, Pierre Ablin, Mathieu Blondel +1
Attention based models such as Transformers involve pairwise interactions between data points, modeled with a learnable attention matrix. Importantly, this attention matrix is norm…
Unsupervised Ground Metric Learning using Wasserstein Singular Vectors
Geert-Jan Huizing, Laura Cantini, Gabriel Peyré
Defining meaningful distances between samples in a dataset is a fundamental problem in machine learning. Optimal Transport (OT) lifts a distance between features (the "ground metri…