Learning with minibatch Wasserstein : asymptotic and gradient properties

Abstract

Optimal transport distances are powerful tools to compare probability distributions and have found many applications in machine learning. Yet their algorithmic complexity prevents their direct use on large scale datasets. To overcome this challenge, practitioners compute these distances on minibatches {\em i.e.} they average the outcome of several smaller optimal transport problems. We propose in this paper an analysis of this practice, which effects are not well understood so far. We notably argue that it is equivalent to an implicit regularization of the original problem, with appealing properties such as unbiased estimators, gradients and a concentration bound around the expectation, but also with defects such as loss of distance property. Along with this theoretical analysis, we also conduct empirical experiments on gradient flows, GANs or color transfer that highlight the practical interest of this strategy.

References in corpus (7)

• DeepJDOT: Deep Joint Distribution Optimal Transport for Unsupervised Domain Adaptation
• Learning with a Wasserstein Loss
• The Cramer Distance as a Solution to Biased Wasserstein Gradients
• Sample Complexity of Sinkhorn divergences
• Interpolating between Optimal Transport and MMD using Sinkhorn Divergences
• Smooth and Sparse Optimal Transport
• Multiscale Strategies for Computing Optimal Transport

Cited by in corpus (5)

• Missing Data Imputation using Optimal Transport
• Scalable Optimal Transport Methods in Machine Learning: A Contemporary Survey
• Graph Matching Optimization Network for Point Cloud Registration
• Generative Modeling with Optimal Transport Maps
• Randomized Stochastic Gradient Descent Ascent