14 citations · 30 across the 11 of their papers we have counts for
11 papers
Abide by the Law and Follow the Flow: Conservation Laws for Gradient Flows
Sibylle Marcotte, Rémi Gribonval, Gabriel Peyré
Understanding the geometric properties of gradient descent dynamics is a key ingredient in deciphering the recent success of very large machine learning models. A striking observat…
Test like you Train in Implicit Deep Learning
Zaccharie Ramzi, Pierre Ablin, Gabriel Peyré +1
Implicit deep learning has recently gained popularity with applications ranging from meta-learning to Deep Equilibrium Networks (DEQs). In its general formulation, it relies on exp…
Fast, Differentiable and Sparse Top-k: a Convex Analysis Perspective
Michael E. Sander, Joan Puigcerver, Josip Djolonga +2
The top-k operator returns a sparse vector, where the non-zero values correspond to the k largest values of the input. Unfortunately, because it is a discontinuous function, it is…
Unbalanced Optimal Transport, from Theory to Numerics
Thibault Séjourné, Gabriel Peyré, François-Xavier Vialard
Optimal Transport (OT) has recently emerged as a central tool in data sciences to compare in a geometrically faithful way point clouds and more generally probability distributions.…
Smooth over-parameterized solvers for non-smooth structured optimization
Clarice Poon, Gabriel Peyré
Non-smooth optimization is a core ingredient of many imaging or machine learning pipelines. Non-smoothness encodes structural constraints on the solutions, such as sparsity, group…
Do Residual Neural Networks discretize Neural Ordinary Differential Equations?
Michael E. Sander, Pierre Ablin, Gabriel Peyré
Neural Ordinary Differential Equations (Neural ODEs) are the continuous analog of Residual Neural Networks (ResNets). We investigate whether the discrete dynamics defined by a ResN…
Faster Unbalanced Optimal Transport: Translation invariant Sinkhorn and 1-D Frank-Wolfe
Thibault Séjourné, François-Xavier Vialard, Gabriel Peyré
Unbalanced optimal transport (UOT) extends optimal transport (OT) to take into account mass variations to compare distributions. This is crucial to make OT successful in ML applica…
Global convergence of ResNets: From finite to infinite width using linear parameterization
Raphaël Barboni, Gabriel Peyré, François-Xavier Vialard
Overparametrization is a key factor in the absence of convexity to explain global convergence of gradient descent (GD) for neural networks. Beside the well studied lazy regime, inf…
Randomized Stochastic Gradient Descent Ascent
Othmane Sebbouh, Marco Cuturi, Gabriel Peyré
An increasing number of machine learning problems, such as robust or adversarial variants of existing algorithms, require minimizing a loss function that is itself defined as a max…
Sinkformers: Transformers with Doubly Stochastic Attention
Michael E. Sander, Pierre Ablin, Mathieu Blondel +1
Attention based models such as Transformers involve pairwise interactions between data points, modeled with a learnable attention matrix. Importantly, this attention matrix is norm…
Unsupervised Ground Metric Learning using Wasserstein Singular Vectors
Geert-Jan Huizing, Laura Cantini, Gabriel Peyré
Defining meaningful distances between samples in a dataset is a fundamental problem in machine learning. Optimal Transport (OT) lifts a distance between features (the "ground metri…