11 papers
Curvature of optimal transport with respect to the cost and applications to inverse optimal transport
Gabriel Peyré, Clarice Poon, Oscar Tron
We study the inverse optimal transport problem of recovering the ground cost from an optimal transport plan. In discrete settings, this problem reduces to inverse linear programmin…
Sample complexity of unbalanced entropic OT
Francisco Andrade, Gabriel Peyré, Clarice Poon
Optimal transport (OT) has become a central language for comparing probability measures, but exact balanced OT is often both too rigid for data with missing, created, or destroyed…
Token Sample Complexity of Attention
Léa Bohbot, Cyril Letrouit, Gabriel Peyré +1
As context windows in large language models continue to expand, it is essential to characterize how attention behaves at extreme sequence lengths. We introduce token sample complex…
Balanced LoRA: Removing Parameter Invariance to Accelerate Convergence
Valérie Castin, Kimia Nadjahi, Pierre Ablin +1
Low-Rank Adaptation (LoRA) is the most widely adopted method for fine-tuning large language models. Notably, LoRA is inherently overparameterized: multiple pairs of low-rank factor…
Optimal and Diffusion Transports in Machine Learning
Gabriel Peyré
Several problems in machine learning are naturally expressed as the design and analysis of time-evolving probability distributions. This includes sampling via diffusion methods, op…
Training Infinitely Deep and Wide Transformers
Raphaël Barboni, Maarten V. de Hoop, Takashi Furuya +1
Transformers have become the dominant architecture in modern machine learning, yet the theoretical understanding of their training dynamics remains limited. This paper develops a r…