Wasserstein GANs Work Because They Fail (to Approximate the Wasserstein Distance)
arXiv:2103.01678
Abstract
Wasserstein GANs are based on the idea of minimising the Wasserstein distance between a real and a generated distribution. We provide an in-depth mathematical analysis of differences between the theoretical setup and the reality of training Wasserstein GANs. In this work, we gather both theoretical and empirical evidence that the WGAN loss is not a meaningful approximation of the Wasserstein distance. Moreover, we argue that the Wasserstein distance is not even a desirable loss function for deep generative models, and conclude that the success of Wasserstein GANs can in truth be attributed to a failure to approximate the Wasserstein distance.
References in corpus (13)
- Scikit-learn: Machine Learning in Python
- Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
- BEGAN: Boundary Equilibrium Generative Adversarial Networks
- The Cramer Distance as a Solution to Biased Wasserstein Gradients
- Sample Complexity of Sinkhorn divergences
- Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances
- Subspace Robust Wasserstein Distances
- Inductive Bias of Deep Convolutional Networks through Pooling Geometry
- Learning Probability Measures with respect to Optimal Transport Metrics
- How Well Do WGANs Estimate the Wasserstein Metric?
- Implicit competitive regularization in GANs
- (q,p)-Wasserstein GANs: Comparing Ground Metrics for Wasserstein GANs
- A gradual, semi-discrete approach to generative network training via explicit Wasserstein minimization
Cited by in corpus (7)
- A Systematic Survey of Regularization and Normalization in GANs
- Scalable Optimal Transport Methods in Machine Learning: A Contemporary Survey
- Do Neural Optimal Transport Solvers Work? A Continuous Wasserstein-2 Benchmark
- Adversarial Intrinsic Motivation for Reinforcement Learning
- Wasserstein GANs with Gradient Penalty Compute Congested Transport
- Manifold-preserved GANs
- Exploiting Chain Rule and Bayes' Theorem to Compare Probability Distributions