6 papers · 1 filter
Curvature of optimal transport with respect to the cost and applications to inverse optimal transport
Gabriel Peyré, Clarice Poon, Oscar Tron
We study the inverse optimal transport problem of recovering the ground cost from an optimal transport plan. In discrete settings, this problem reduces to inverse linear programmin…
Optimal and Diffusion Transports in Machine Learning
Gabriel Peyré
Several problems in machine learning are naturally expressed as the design and analysis of time-evolving probability distributions. This includes sampling via diffusion methods, op…
Training Infinitely Deep and Wide Transformers
Raphaël Barboni, Maarten V. de Hoop, Takashi Furuya +1
Transformers have become the dominant architecture in modern machine learning, yet the theoretical understanding of their training dynamics remains limited. This paper develops a r…
On the global convergence of gradient flow for wide shallow models beyond homogeneous nonlinearities
Romain Petit, Clarice Poon, Gabriel Peyré +1
A surprising phenomenon in the training of neural networks is the ability of gradient descent to find global minimizers of the training loss despite its non-convexity. Following ea…
Muon Dynamics as a Spectral Wasserstein Flow
Gabriel Peyré
Gradient normalization stabilizes deep-learning optimization, and spectral normalizations are especially natural for matrix-shaped parameter blocks; Muon is the motivating example.…
Robust Sublinear Convergence Rates for Iterative Bregman Projections
Gabriel Peyré
Entropic regularization provides a simple way to approximate linear programs whose constraints split into two or more tractable blocks. The resulting objectives are amenable to cyc…