How Well Do WGANs Estimate the Wasserstein Metric?

Abstract

Generative modelling is often cast as minimizing a similarity measure between a data distribution and a model distribution. Recently, a popular choice for the similarity measure has been the Wasserstein metric, which can be expressed in the Kantorovich duality formulation as the optimum difference of the expected values of a potential function under the real data distribution and the model hypothesis. In practice, the potential is approximated with a neural network and is called the discriminator. Duality constraints on the function class of the discriminator are enforced approximately, and the expectations are estimated from samples. This gives at least three sources of errors: the approximated discriminator and constraints, the estimation of the expectation value, and the optimization required to find the optimal potential. In this work, we study how well the methods, that are used in generative adversarial networks to approximate the Wasserstein metric, perform. We consider, in particular, the $c$-transform formulation, which eliminates the need to enforce the constraints explicitly. We demonstrate that the $c$-transform allows for a more accurate estimation of the true Wasserstein metric from samples, but surprisingly, does not perform the best in the generative setting.

arXiv comment23 pages

References in corpus (7)

• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Progressive Growing of GANs for Improved Quality, Stability, and Variation
• Sample Complexity of Sinkhorn divergences
• Interpolating between Optimal Transport and MMD using Sinkhorn Divergences
• Unnormalized Optimal Transport
• Statistical bounds for entropic optimal transport: sample complexity and the central limit theorem
• (q,p)-Wasserstein GANs: Comparing Ground Metrics for Wasserstein GANs

Cited by in corpus (9)

• Wasserstein GANs for MR Imaging: from Paired to Unpaired Training
• A Systematic Survey of Regularization and Normalization in GANs
• Wasserstein GANs Work Because They Fail (to Approximate the Wasserstein Distance)
• Do Neural Optimal Transport Solvers Work? A Continuous Wasserstein-2 Benchmark
• Wasserstein GANs with Gradient Penalty Compute Congested Transport
• Training Wasserstein GANs without gradient penalties
• Exploiting Chain Rule and Bayes' Theorem to Compare Probability Distributions
• Convergence and finite sample approximations of entropic regularized Wasserstein distances in Gaussian and RKHS settings
• A method for quantifying the generalization capabilities of generative models for solving Ising models