Unbalanced minibatch Optimal Transport; applications to Domain Adaptation

Abstract

Optimal transport distances have found many applications in machine learning for their capacity to compare non-parametric probability distributions. Yet their algorithmic complexity generally prevents their direct use on large scale datasets. Among the possible strategies to alleviate this issue, practitioners can rely on computing estimates of these distances over subsets of data, {\em i.e.} minibatches. While computationally appealing, we highlight in this paper some limits of this strategy, arguing it can lead to undesirable smoothing effects. As an alternative, we suggest that the same minibatch strategy coupled with unbalanced optimal transport can yield more robust behavior. We discuss the associated theoretical properties, such as unbiased estimators, existence of gradients and concentration bounds. Our experimental study shows that in challenging problems associated to domain adaptation, the use of unbalanced optimal transport leads to significantly better results, competing with or surpassing recent baselines.

References in corpus (18)

• Conditional Adversarial Domain Adaptation
• When Does Label Smoothing Help?
• VisDA: The Visual Domain Adaptation Challenge
• DeepJDOT: Deep Joint Distribution Optimal Transport for Unsupervised Domain Adaptation
• Wasserstein Distance Guided Representation Learning for Domain Adaptation
• The Cramer Distance as a Solution to Biased Wasserstein Gradients
• Sample Complexity of Sinkhorn divergences
• Subspace Robust Wasserstein Distances
• Missing Data Imputation using Optimal Transport
• Sinkhorn Divergences for Unbalanced Optimal Transport
• An Entropic Optimal Transport Loss for Learning Deep Neural Networks under Label Noise in Remote Sensing Images
• Robust Optimal Transport with Applications in Generative Modeling and Domain Adaptation
• On Unbalanced Optimal Transport: An Analysis of Sinkhorn Algorithm
• Analysis of nonsmooth stochastic approximation: the differential inclusion approach
• Minibatch optimal transport distances; analysis and applications
• Outlier-Robust Optimal Transport
• Adversarial-Learned Loss for Domain Adaptation
• On properties of the Generalized Wasserstein distance

Cited by in corpus (5)

• TimeMatch: Unsupervised Cross-Region Adaptation by Temporal Shift Estimation
• Unsupervised Domain Adaptation for LiDAR Panoptic Segmentation
• SALUDA: Surface-based Automotive Lidar Unsupervised Domain Adaptation
• Scalable Optimal Transport Methods in Machine Learning: A Contemporary Survey
• Prototypical Partial Optimal Transport for Universal Domain Adaptation