Robust Optimal Transport with Applications in Generative Modeling and Domain Adaptation

Abstract

Optimal Transport (OT) distances such as Wasserstein have been used in several areas such as GANs and domain adaptation. OT, however, is very sensitive to outliers (samples with large noise) in the data since in its objective function, every sample, including outliers, is weighed similarly due to the marginal constraints. To remedy this issue, robust formulations of OT with unbalanced marginal constraints have previously been proposed. However, employing these methods in deep learning problems such as GANs and domain adaptation is challenging due to the instability of their dual optimization solvers. In this paper, we resolve these issues by deriving a computationally-efficient dual form of the robust OT optimization that is amenable to modern deep learning applications. We demonstrate the effectiveness of our formulation in two applications of GANs and domain adaptation. Our approach can train state-of-the-art GAN models on noisy datasets corrupted with outlier distributions. In particular, our optimization computes weights for training samples reflecting how difficult it is for those samples to be generated in the model. In domain adaptation, our robust OT formulation leads to improved accuracy compared to the standard adversarial adaptation methods. Our code is available at https://github.com/yogeshbalaji/robustOT.

arXiv commentAccepted in NeurIPS 2020. Code available at https://github.com/yogeshbalaji/robustOT

References in corpus (10)

• Learning Transferable Features with Deep Adaptation Networks
• Conditional Adversarial Domain Adaptation
• Unsupervised Domain Adaptation with Residual Transfer Networks
• VisDA: The Visual Domain Adaptation Challenge
• Learning with a Wasserstein Loss
• Wasserstein Distance Guided Representation Learning for Domain Adaptation
• Maximum Classifier Discrepancy for Unsupervised Domain Adaptation
• Scaling Algorithms for Unbalanced Transport Problems
• Moment Matching for Multi-Source Domain Adaptation
• Scalable Unbalanced Optimal Transport using Generative Adversarial Networks

Cited by in corpus (10)

• Unbalanced minibatch Optimal Transport; applications to Domain Adaptation
• Outlier-Robust Optimal Transport
• On Robust Optimal Transport: Computational Complexity and Barycenter Computation
• Faster Unbalanced Optimal Transport: Translation invariant Sinkhorn and 1-D Frank-Wolfe
• Advancing Prompt Learning through an External Layer
• Outlier-Robust Optimal Transport: Duality, Structure, and Statistical Analysis
• A novel notion of barycenter for probability distributions based on optimal weak mass transport
• A Pseudo-Metric between Probability Distributions based on Depth-Trimmed Regions
• On Multimarginal Partial Optimal Transport: Equivalent Forms and Computational Complexity
• Explanatory Model Monitoring to Understand the Effects of Feature Shifts on Performance