Improving Mini-batch Optimal Transport via Partial Transportation
arXiv:2108.09645
Abstract
Mini-batch optimal transport (m-OT) has been widely used recently to deal with the memory issue of OT in large-scale applications. Despite their practicality, m-OT suffers from misspecified mappings, namely, mappings that are optimal on the mini-batch level but are partially wrong in the comparison with the optimal transportation plan between the original measures. Motivated by the misspecified mappings issue, we propose a novel mini-batch method by using partial optimal transport (POT) between mini-batch empirical measures, which we refer to as mini-batch partial optimal transport (m-POT). Leveraging the insight from the partial transportation, we explain the source of misspecified mappings from the m-OT and motivate why limiting the amount of transported masses among mini-batches via POT can alleviate the incorrect mappings. Finally, we carry out extensive experiments on various applications such as deep domain adaptation, partial domain adaptation, deep generative model, color transfer, and gradient flow to demonstrate the favorable performance of m-POT compared to current mini-batch methods.
arXiv commentAccepted to ICML 2022, 36 pages, 18 figures, 18 tables
References in corpus (11)
- GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
- VisDA: The Visual Domain Adaptation Challenge
- DeepJDOT: Deep Joint Distribution Optimal Transport for Unsupervised Domain Adaptation
- Near-linear time approximation algorithms for optimal transport via Sinkhorn iteration
- On Efficient Optimal Transport: An Analysis of Greedy and Accelerated Mirror Descent Algorithms
- Multilevel Clustering via Wasserstein Means
- On Unbalanced Optimal Transport: An Analysis of Sinkhorn Algorithm
- Minibatch optimal transport distances; analysis and applications
- Adversarial Computation of Optimal Transport Maps
- Probabilistic Multilevel Clustering via Composite Transportation Distance
- Partial Optimal Transport with Applications on Positive-Unlabeled Learning