On Convergence and Stability of GANs

Abstract

We propose studying GAN training dynamics as regret minimization, which is in contrast to the popular view that there is consistent minimization of a divergence between real and generated distributions. We analyze the convergence of GAN training from this new point of view to understand why mode collapse happens. We hypothesize the existence of undesirable local equilibria in this non-convex game to be responsible for mode collapse. We observe that these local equilibria often exhibit sharp gradients of the discriminator function around some real data points. We demonstrate that these degenerate local equilibria can be avoided with a gradient penalty scheme called DRAGAN. We show that DRAGAN enables faster training, achieves improved stability with fewer mode collapses, and leads to generator networks with better modeling performance across a variety of architectures and objective functions.

Analysis of convergence and mode collapse by studying GAN training process as regret minimization. Some new results

References in corpus (4)

• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• NIPS 2016 Tutorial: Generative Adversarial Networks
• Energy-based Generative Adversarial Network
• Mode Regularized Generative Adversarial Networks

Cited by in corpus (105)

• ResViT: Residual vision transformers for multi-modal medical image synthesis
• Generative Adversarial Networks in Computer Vision: A Survey and Taxonomy
• GANSynth: Adversarial Neural Audio Synthesis
• Enforcing Statistical Constraints in Generative Adversarial Networks for Modeling Chaotic Dynamical Systems
• Towards the Automatic Anime Characters Creation with Generative Adversarial Networks
• ContraGAN: Contrastive Learning for Conditional Image Generation
• Diagnosing and Enhancing VAE Models
• ViTGAN: Training GANs with Vision Transformers
• Sorting out Lipschitz function approximation
• Stabilizing Generative Adversarial Networks: A Survey
• Dropout-GAN: Learning from a Dynamic Ensemble of Discriminators
• Consistency Regularization for Generative Adversarial Networks
• Lipschitz Generative Adversarial Nets
• RELATE: Physically Plausible Multi-Object Scene Synthesis Using Structured Latent Spaces
• DiverGAN: An Efficient and Effective Single-Stage Framework for Diverse Text-to-Image Generation
• A Large-Scale Study on Regularization and Normalization in GANs
• The Information Autoencoding Family: A Lagrangian Perspective on Latent Variable Generative Models
• PConv: Simple yet Effective Convolutional Layer for Generative Adversarial Network
• FedGAN: Federated Generative Adversarial Networks for Distributed Data
• Adversarial score matching and improved sampling for image generation
• Synthesis of Realistic ECG using Generative Adversarial Networks
• Feature Quantization Improves GAN Training
• Global Texture Enhancement for Fake Face Detection in the Wild
• Improved Consistency Regularization for GANs
• GANs May Have No Nash Equilibria
• Regularization Methods for Generative Adversarial Networks: An Overview of Recent Studies
• Semi-supervised Image Attribute Editing using Generative Adversarial Networks
• On Catastrophic Forgetting and Mode Collapse in Generative Adversarial Networks
• Smoothness and Stability in GANs
• Encoding Invariances in Deep Generative Models
• Some Theoretical Insights into Wasserstein GANs
• A case for new neural network smoothness constraints
• Synthesizing lesions using contextual GANs improves breast cancer classification on mammograms
• An error analysis of generative adversarial networks for learning distributions
• Adversarial Computation of Optimal Transport Maps
• Implicit competitive regularization in GANs
• Twin-GAN -- Unpaired Cross-Domain Image Translation with Weight-Sharing GANs
• Puzzle-AE: Novelty Detection in Images through Solving Puzzles
• Adversarial symmetric GANs: bridging adversarial samples and adversarial networks
• Quantitatively Evaluating GANs With Divergences Proposed for Training
• Statistical guarantees for generative models without domination
• Aggregated Multi-GANs for Controlled 3D Human Motion Prediction
• Regularizing Generative Adversarial Networks under Limited Data
• Generative Convolution Layer for Image Generation
• Scalable Bayesian Inverse Reinforcement Learning
• Using generative adversarial networks to synthesize artificial financial datasets
• Alleviating Mode Collapse in GAN via Diversity Penalty Module
• Towards Efficient and Unbiased Implementation of Lipschitz Continuity in GANs
• AlphaGAN: Fully Differentiable Architecture Search for Generative Adversarial Networks
• An Improved Self-supervised GAN via Adversarial Training
• Rates of convergence for density estimation with generative adversarial networks
• Competing Against Equilibria in Zero-Sum Games with Evolving Payoffs
• Non-saturating GAN training as divergence minimization
• Understanding the Effectiveness of Lipschitz-Continuity in Generative Adversarial Nets
• Convolutional Normalization: Improving Deep Convolutional Network Robustness and Training
• Generative Minimization Networks: Training GANs Without Competition
• Stable Policy Optimization via Off-Policy Divergence Regularization
• GANs beyond divergence minimization
• TwoStreamVAN: Improving Motion Modeling in Video Generation
• Generative Modeling with Optimal Transport Maps
• CatVRNN: Generating Category Texts via Multi-task Learning
• Max-Affine Spline Insights into Deep Generative Networks
• Poincaré Recurrence, Cycles and Spurious Equilibria in Gradient-Descent-Ascent for Non-Convex Non-Concave Zero-Sum Games
• Learning in Non-convex Games with an Optimization Oracle
• PATE-AAE: Incorporating Adversarial Autoencoder into Private Aggregation of Teacher Ensembles for Spoken Command Classification
• Approximating Lipschitz continuous functions with GroupSort neural networks
• Generalized Energy Based Models
• Augmented Cyclic Consistency Regularization for Unpaired Image-to-Image Translation
• SDA-GAN: Unsupervised Image Translation Using Spectral Domain Attention-Guided Generative Adversarial Network
• Learning Disentangled Representations with Latent Variation Predictability
• Evaluating Generative Adversarial Networks on Explicitly Parameterized Distributions
• Gradient penalty from a maximum margin perspective
• Training Wasserstein GANs without gradient penalties
• Collaborative Sampling in Generative Adversarial Networks
• Blur, Noise, and Compression Robust Generative Adversarial Networks
• Adversarial-Based Knowledge Distillation for Multi-Model Ensemble and Noisy Data Refinement
• A Differential Game Theoretic Neural Optimizer for Training Residual Networks
• MaGNET: Uniform Sampling from Deep Generative Network Manifolds Without Retraining
• Game of GANs: Game-Theoretical Models for Generative Adversarial Networks
• Online Kernel based Generative Adversarial Networks
• Adaptive Divergence for Rapid Adversarial Optimization
• Conditional Generative Modeling via Learning the Latent Space
• Conditioning Trick for Training Stable GANs
• Fair Adversarial Networks
• On reproduction of On the regularization of Wasserstein GANs
• Exploiting Chain Rule and Bayes' Theorem to Compare Probability Distributions
• Optimal Transport Based Generative Autoencoders
• Latent Space Arc Therapy Optimization
• When Relation Networks meet GANs: Relation GANs with Triplet Loss
• Multiclass non-Adversarial Image Synthesis, with Application to Classification from Very Small Sample
• Analysis of Discriminator in RKHS Function Space for Kullback-Leibler Divergence Estimation
• An Empirical Study of the Effects of Sample-Mixing Methods for Efficient Training of Generative Adversarial Networks
• Follow the Neurally-Perturbed Leader for Adversarial Training
• Noise Robust Generative Adversarial Networks
• KL Guided Domain Adaptation
• Fuzzy Generative Adversarial Networks
• A deep learning framework for morphologic detail beyond the diffraction limit in infrared spectroscopic imaging
• Segmentation and Generation of Magnetic Resonance Images by Deep Neural Networks
• The Geometric Occam's Razor Implicit in Deep Learning
• Editable Generative Adversarial Networks: Generating and Editing Faces Simultaneously
• Generative Adversarial Networks for Unsupervised Object Co-localization
• Neuronal Learning Analysis using Cycle-Consistent Adversarial Networks
• ReshapeGAN: Object Reshaping by Providing A Single Reference Image
• FusedProp: Towards Efficient Training of Generative Adversarial Networks
• Towards GANs' Approximation Ability