Some Theoretical Insights into Wasserstein GANs

Abstract

Generative Adversarial Networks (GANs) have been successful in producing outstanding results in areas as diverse as image, video, and text generation. Building on these successes, a large number of empirical studies have validated the benefits of the cousin approach called Wasserstein GANs (WGANs), which brings stabilization in the training process. In the present paper, we add a new stone to the edifice by proposing some theoretical advances in the properties of WGANs. First, we properly define the architecture of WGANs in the context of integral probability metrics parameterized by neural networks and highlight some of their basic mathematical features. We stress in particular interesting optimization properties arising from the use of a parametric 1-Lipschitz discriminator. Then, in a statistically-driven approach, we study the convergence of empirical WGANs as the sample size tends to infinity, and clarify the adversarial effects of the generator and the discriminator by underlining some trade-off properties. These features are finally illustrated with experiments using both synthetic and real-world datasets.

References in corpus (26)

• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Spectral Normalization for Generative Adversarial Networks
• On the Number of Linear Regions of Deep Neural Networks
• f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization
• MMD GAN: Towards Deeper Understanding of Moment Matching Network
• Unrolled Generative Adversarial Networks
• C-RNN-GAN: Continuous recurrent neural networks with adversarial training
• MaskGAN: Better Text Generation via Filling in the______
• Improved Precision and Recall Metric for Assessing Generative Models
• On Convergence and Stability of GANs
• Stabilizing Training of Generative Adversarial Networks through Regularization
• Generalization and Equilibrium in Generative Adversarial Nets (GANs)
• On the regularization of Wasserstein GANs
• On the number of response regions of deep feed forward networks with piece-wise linear activations
• Sorting out Lipschitz function approximation
• Approximation and Convergence Properties of Generative Adversarial Learning
• A Short Note on Concentration Inequalities for Random Vectors with SubGaussian Norm
• Towards High Resolution Video Generation with Progressive Growing of Sliced Wasserstein GANs
• Concentration in unbounded metric spaces and algorithmic stability
• Lipschitz Generative Adversarial Nets
• Statistical guarantees for generative models without domination
• Generalization Properties of Optimal Transport GANs with Latent Distribution Learning
• Norm-preserving Orthogonal Permutation Linear Unit Activation Functions (OPLU)
• Rates of convergence for density estimation with generative adversarial networks

Cited by in corpus (6)

• Ten Years of Generative Adversarial Nets (GANs): A survey of the state-of-the-art
• Statistical guarantees for generative models without domination
• Rates of convergence for density estimation with generative adversarial networks
• Statistical analysis of Wasserstein GANs with applications to time series forecasting
• Optimal 1-Wasserstein Distance for WGANs
• Statistically Optimal Generative Modeling with Maximum Deviation from the Empirical Distribution