Normalized Wasserstein Distance for Mixture Distributions with Applications in Adversarial Learning and Domain Adaptation
arXiv:1902.00415
Abstract
Understanding proper distance measures between distributions is at the core of several learning tasks such as generative models, domain adaptation, clustering, etc. In this work, we focus on mixture distributions that arise naturally in several application domains where the data contains different sub-populations. For mixture distributions, established distance measures such as the Wasserstein distance do not take into account imbalanced mixture proportions. Thus, even if two mixture distributions have identical mixture components but different mixture proportions, the Wasserstein distance between them will be large. This often leads to undesired results in distance-based learning methods for mixture distributions. In this paper, we resolve this issue by introducing the Normalized Wasserstein measure. The key idea is to introduce mixture proportions as optimization variables, effectively normalizing mixture proportions in the Wasserstein formulation. Using the proposed normalized Wasserstein measure leads to significant performance gains for mixture distributions with imbalanced mixture proportions compared to the vanilla Wasserstein distance. We demonstrate the effectiveness of the proposed measure in GANs, domain adaptation and adversarial clustering in several benchmark datasets.
arXiv commentAccepted at ICCV 2019
References in corpus (9)
- Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
- Large Scale GAN Training for High Fidelity Natural Image Synthesis
- Learning Transferable Features with Deep Adaptation Networks
- Improved Training of Wasserstein GANs
- Unsupervised Domain Adaptation with Residual Transfer Networks
- VisDA: The Visual Domain Adaptation Challenge
- Generative Moment Matching Networks
- Least Squares Generative Adversarial Networks
- Analysis of Kernel Mean Matching under Covariate Shift