Understanding the Effectiveness of Lipschitz-Continuity in Generative Adversarial Nets

Abstract

In this paper, we investigate the underlying factor that leads to failure and success in the training of GANs. We study the property of the optimal discriminative function and show that in many GANs, the gradient from the optimal discriminative function is not reliable, which turns out to be the fundamental cause of failure in training of GANs. We further demonstrate that a well-defined distance metric does not necessarily guarantee the convergence of GANs. Finally, we prove in this paper that Lipschitz-continuity condition is a general solution to make the gradient of the optimal discriminative function reliable, and characterized the necessary condition where Lipschitz-continuity ensures the convergence, which leads to a broad family of valid GAN objectives under Lipschitz-continuity condition, where Wasserstein distance is one special case. We experiment with several new objectives, which are sound according to our theorems, and we found that, compared with Wasserstein distance, the outputs of the discriminator with new objectives are more stable and the final qualities of generated samples are also consistently higher than those produced by Wasserstein distance.

References in corpus (29)

• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Spectral Normalization for Generative Adversarial Networks
• Self-Attention Generative Adversarial Networks
• Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks
• Conditional Image Synthesis With Auxiliary Classifier GANs
• Progressive Growing of GANs for Improved Quality, Stability, and Variation
• Improved Training of Wasserstein GANs
• NIPS 2016 Tutorial: Generative Adversarial Networks
• Energy-based Generative Adversarial Network
• Which Training Methods for GANs do actually Converge?
• MMD GAN: Towards Deeper Understanding of Moment Matching Network
• Unrolled Generative Adversarial Networks
• Unsupervised and Semi-supervised Learning with Categorical Generative Adversarial Networks
• SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient
• Neural Photo Editing with Introspective Adversarial Networks
• Mode Regularized Generative Adversarial Networks
• On Convergence and Stability of GANs
• The Numerics of GANs
• Plug & Play Generative Networks: Conditional Iterative Generation of Images in Latent Space
• Least Squares Generative Adversarial Networks
• On the regularization of Wasserstein GANs
• Do GANs actually learn the distribution? An empirical study
• Many Paths to Equilibrium: GANs Do Not Need to Decrease a Divergence At Every Step
• Large-Scale Optimal Transport and Mapping Estimation
• Geometric GAN
• Generative Adversarial Nets from a Density Ratio Estimation Perspective
• Is Generator Conditioning Causally Related to GAN Performance?
• Activation Maximization Generative Adversarial Nets
• Distributional Adversarial Networks

Cited by in corpus (5)

• On Self Modulation for Generative Adversarial Networks
• Improving GAN Training with Probability Ratio Clipping and Sample Reweighting
• Self-Supervised GAN to Counter Forgetting
• Adversarial Networks and Autoencoders: The Primal-Dual Relationship and Generalization Bounds
• General Probabilistic Surface Optimization and Log Density Estimation