Generative Adversarial Network Training is a Continual Learning Problem

Abstract

Generative Adversarial Networks (GANs) have proven to be a powerful framework for learning to draw samples from complex distributions. However, GANs are also notoriously difficult to train, with mode collapse and oscillations a common problem. We hypothesize that this is at least in part due to the evolution of the generator distribution and the catastrophic forgetting tendency of neural networks, which leads to the discriminator losing the ability to remember synthesized samples from previous instantiations of the generator. Recognizing this, our contributions are twofold. First, we show that GAN training makes for a more interesting and realistic benchmark for continual learning methods evaluation than some of the more canonical datasets. Second, we propose leveraging continual learning techniques to augment the discriminator, preserving its ability to recognize previous generator samples. We show that the resulting methods add only a light amount of computation, involve minimal changes to the model, and result in better overall performance on the examined image and text generation tasks.

References in corpus (8)

• Categorical Reparameterization with Gumbel-Softmax
• Improved Training of Wasserstein GANs
• Improved Techniques for Training GANs
• VEEGAN: Reducing Mode Collapse in GANs using Implicit Variational Learning
• Long Text Generation via Adversarial Training with Leaked Information
• Adversarial Feature Matching for Text Generation
• Continual Learning in Generative Adversarial Nets
• Gradient descent GAN optimization is locally stable

Cited by in corpus (15)

• A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
• Review: Deep Learning in Electron Microscopy
• Feature Quantization Improves GAN Training
• Audio inpainting with generative adversarial network
• On Catastrophic Forgetting and Mode Collapse in Generative Adversarial Networks
• Continual Learning using a Bayesian Nonparametric Dictionary of Weight Factors
• Self-Supervised GAN to Counter Forgetting
• Understanding and Stabilizing GANs' Training Dynamics with Control Theory
• Self-Supervised GANs via Auxiliary Rotation Loss
• FastVC: Fast Voice Conversion with non-parallel data
• Meta-Learned Attribute Self-Gating for Continual Generalized Zero-Shot Learning
• Frosting Weights for Better Continual Training
• Improving Text Generation with Student-Forcing Optimal Transport
• Association: Remind Your GAN not to Forget
• IB-DRR: Incremental Learning with Information-Back Discrete Representation Replay