Improving Generalization and Stability of Generative Adversarial Networks

Abstract

Generative Adversarial Networks (GANs) are one of the most popular tools for learning complex high dimensional distributions. However, generalization properties of GANs have not been well understood. In this paper, we analyze the generalization of GANs in practical settings. We show that discriminators trained on discrete datasets with the original GAN loss have poor generalization capability and do not approximate the theoretically optimal discriminator. We propose a zero-centered gradient penalty for improving the generalization of the discriminator by pushing it toward the optimal discriminator. The penalty guarantees the generalization and convergence of GANs. Experiments on synthetic and large scale datasets verify our theoretical analysis.

ICLR 2019

Cited by in corpus (29)

• A survey on GANs for computer vision: Recent research, analysis and taxonomy
• Stabilizing Generative Adversarial Networks: A Survey
• A Systematic Survey of Regularization and Normalization in GANs
• Machine Learning for Physics-Informed Generation of Dispersed Multiphase Flow Using Generative Adversarial Networks
• On Catastrophic Forgetting and Mode Collapse in Generative Adversarial Networks
• Variational Dynamic for Self-Supervised Exploration in Deep Reinforcement Learning
• GNPassGAN: Improved Generative Adversarial Networks For Trawling Offline Password Guessing
• An error analysis of generative adversarial networks for learning distributions
• Unsupervised anomaly localization using VAE and beta-VAE
• Training Federated GANs with Theoretical Guarantees: A Universal Aggregation Approach
• Train simultaneously, generalize better: Stability of gradient-based minimax learners
• Bounds all around: training energy-based models with bidirectional bounds
• From Shadow Segmentation to Shadow Removal
• SDE approximations of GANs training and its long-run behavior
• Slimmable Generative Adversarial Networks
• Alleviation of Gradient Exploding in GANs: Fake Can Be Real
• A Novel Framework for Selection of GANs for an Application
• Adversarial Imitation Learning with Trajectorial Augmentation and Correction
• Semantically Robust Unpaired Image Translation for Data with Unmatched Semantics Statistics
• Conditioning Trick for Training Stable GANs
• Conditional Generative Modeling via Learning the Latent Space
• Semisupervised Adversarial Neural Networks for Cyber Security Transfer Learning
• Toward a Generalization Metric for Deep Generative Models
• Semantic Preserving Generative Adversarial Models
• Improving Generative Adversarial Networks with Local Coordinate Coding
• Generative Adversarial Network for Probabilistic Forecast of Random Dynamical System
• Reliable Estimation of Individual Treatment Effect with Causal Information Bottleneck
• When Relation Networks meet GANs: Relation GANs with Triplet Loss
• Analysis of Discriminator in RKHS Function Space for Kullback-Leibler Divergence Estimation