Stabilizing Generative Adversarial Networks: A Survey

Abstract

Generative Adversarial Networks (GANs) are a type of generative model which have received much attention due to their ability to model complex real-world data. Despite their recent successes, the process of training GANs remains challenging, suffering from instability problems such as non-convergence, vanishing or exploding gradients, and mode collapse. In recent years, a diverse set of approaches have been proposed which focus on stabilizing the GAN training procedure. The purpose of this survey is to provide a comprehensive overview of the GAN training stabilization methods which can be found in the literature. We discuss the advantages and disadvantages of each approach, offer a comparative summary, and conclude with a discussion of open problems.

References in corpus (49)

• Auto-Encoding Variational Bayes
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Generative Adversarial Networks: An Overview
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Self-Attention Generative Adversarial Networks
• Generative Adversarial Text to Image Synthesis
• Conditional Image Synthesis With Auxiliary Classifier GANs
• Prioritized Experience Replay
• Improved Training of Wasserstein GANs
• Improved Techniques for Training GANs
• Which Training Methods for GANs do actually Converge?
• BEGAN: Boundary Equilibrium Generative Adversarial Networks
• f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization
• Autonomous Agents Modelling Other Agents: A Comprehensive Survey and Open Problems
• MMD GAN: Towards Deeper Understanding of Moment Matching Network
• A note on the evaluation of generative models
• Unrolled Generative Adversarial Networks
• How Generative Adversarial Networks and Their Variants Work: An Overview
• Generative Moment Matching Networks
• SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient
• Real-valued (Medical) Time Series Generation with Recurrent Conditional GANs
• The Cramer Distance as a Solution to Biased Wasserstein Gradients
• Variational Approaches for Auto-Encoding Generative Adversarial Networks
• On Convergence and Stability of GANs
• A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Models
• Stabilizing Training of Generative Adversarial Networks through Regularization
• How (not) to Train your Generative Model: Scheduled Sampling, Likelihood, Adversary?
• The Numerics of GANs
• AdaGAN: Boosting Generative Models
• The Mechanics of n-Player Differentiable Games
• Dealing with Non-Stationarity in Multi-Agent Deep Reinforcement Learning
• On integral probability metrics, ϕ-divergences and binary classification
• Improving Generalization and Stability of Generative Adversarial Networks
• Improved generator objectives for GANs
• Competitive Gradient Descent
• Convergence Problems with Generative Adversarial Networks (GANs)
• Bayesian GAN
• Generative Adversarial Parallelization
• Multi-objective training of Generative Adversarial Networks with multiple discriminators
• Gang of GANs: Generative Adversarial Networks with Maximum Margin Ranking
• Efficient Regret Minimization in Non-Convex Games
• Evolutionary Generative Adversarial Networks
• GANGs: Generative Adversarial Network Games
• Message Passing Multi-Agent GANs
• Finding Mixed Nash Equilibria of Generative Adversarial Networks
• Implicit competitive regularization in GANs
• Fictitious GAN: Training GANs with Historical Models
• GANs beyond divergence minimization

Cited by in corpus (16)

• Review: Deep Learning in Electron Microscopy
• Inverse design optimization framework via a two-step deep learning approach: application to a wind turbine airfoil
• Compressing PDF sets using generative adversarial networks
• Machine-Learning-Assisted Photonic Device Development: A Multiscale Approach from Theory to Characterization
• GraN-GAN: Piecewise Gradient Normalization for Generative Adversarial Networks
• HAD-Net: A Hierarchical Adversarial Knowledge Distillation Network for Improved Enhanced Tumour Segmentation Without Post-Contrast Images
• DAG-WGAN: Causal Structure Learning With Wasserstein Generative Adversarial Networks
• Sliced Iterative Normalizing Flows
• SDE approximations of GANs training and its long-run behavior
• A Novel Framework for Selection of GANs for an Application
• Game of GANs: Game-Theoretical Models for Generative Adversarial Networks
• Synthetic Data Augmentation for Enhancing Harmful Algal Bloom Detection with Machine Learning
• Generative networks as inverse problems with fractional wavelet scattering networks
• Generative Adversarial Network: Some Analytical Perspectives
• Causality Learning With Wasserstein Generative Adversarial Networks
• Dynamic Flows on Curved Space Generated by Labeled Data