Relaxed Wasserstein with Applications to GANs

Abstract

Wasserstein Generative Adversarial Networks (WGANs) provide a versatile class of models, which have attracted great attention in various applications. However, this framework has two main drawbacks: (i) Wasserstein-1 (or Earth-Mover) distance is restrictive such that WGANs cannot always fit data geometry well; (ii) It is difficult to achieve fast training of WGANs. In this paper, we propose a new class of \textit{Relaxed Wasserstein} (RW) distances by generalizing Wasserstein-1 distance with Bregman cost functions. We show that RW distances achieve nice statistical properties while not sacrificing the computational tractability. Combined with the GANs framework, we develop Relaxed WGANs (RWGANs) which are not only statistically flexible but can be approximated efficiently using heuristic approaches. Experiments on real images demonstrate that the RWGAN with Kullback-Leibler (KL) cost function outperforms other competing approaches, e.g., WGANs, even with gradient penalty.

arXiv commentAccepted by ICASSP 2021; add the references

References in corpus (8)

• Conditional Generative Adversarial Nets
• Generative Adversarial Networks: An Overview
• Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
• NIPS 2016 Tutorial: Generative Adversarial Networks
• Energy-based Generative Adversarial Network
• Towards Principled Methods for Training Generative Adversarial Networks
• Convergence and Concentration of Empirical Measures under Wasserstein Distance in Unbounded Functional Spaces
• On the Limitations of First-Order Approximation in GAN Dynamics

Cited by in corpus (4)

• Understanding GANs: the LQG Setting
• Mode Collapse and Regularity of Optimal Transportation Maps
• Sliced-Wasserstein Flows: Nonparametric Generative Modeling via Optimal Transport and Diffusions
• Generative Adversarial Network: Some Analytical Perspectives