Continual Learning in Generative Adversarial Nets

Abstract

Developments in deep generative models have allowed for tractable learning of high-dimensional data distributions. While the employed learning procedures typically assume that training data is drawn i.i.d. from the distribution of interest, it may be desirable to model distinct distributions which are observed sequentially, such as when different classes are encountered over time. Although conditional variations of deep generative models permit multiple distributions to be modeled by a single network in a disentangled fashion, they are susceptible to catastrophic forgetting when the distributions are encountered sequentially. In this paper, we adapt recent work in reducing catastrophic forgetting to the task of training generative adversarial networks on a sequence of distinct distributions, enabling continual generative modeling.

References in corpus (3)

• Distilling the Knowledge in a Neural Network
• Conditional Generative Adversarial Nets
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks

Cited by in corpus (33)

• A Comprehensive Study of Class Incremental Learning Algorithms for Visual Tasks
• Lifelong Teacher-Student Network Learning
• Memory Replay GANs: learning to generate images from new categories without forgetting
• Triple Memory Networks: a Brain-Inspired Method for Continual Learning
• A Strategy for an Uncompromising Incremental Learner
• Generative Adversarial Network Training is a Continual Learning Problem
• EEC: Learning to Encode and Regenerate Images for Continual Learning
• On Catastrophic Forgetting and Mode Collapse in Generative Adversarial Networks
• Partitioned Variational Inference: A unified framework encompassing federated and continual learning
• Rainbow Memory: Continual Learning with a Memory of Diverse Samples
• Carousel Memory: Rethinking the Design of Episodic Memory for Continual Learning
• Continual Learning with Knowledge Transfer for Sentiment Classification
• Self-Supervised GAN to Counter Forgetting
• Storing Encoded Episodes as Concepts for Continual Learning
• Self-Supervised GANs via Auxiliary Rotation Loss
• GAN Cocktail: mixing GANs without dataset access
• Continual Learning: Tackling Catastrophic Forgetting in Deep Neural Networks with Replay Processes
• GAN Memory with No Forgetting
• Reviewing continual learning from the perspective of human-level intelligence
• Frosting Weights for Better Continual Training
• Incremental Embedding Learning via Zero-Shot Translation
• Learn distributed GAN with Temporary Discriminators
• DO-GAN: A Double Oracle Framework for Generative Adversarial Networks
• I3DOL: Incremental 3D Object Learning without Catastrophic Forgetting
• Incremental Class Learning using Variational Autoencoders with Similarity Learning
• Lifelong Learning Process: Self-Memory Supervising and Dynamically Growing Networks
• Piggyback GAN: Efficient Lifelong Learning for Image Conditioned Generation
• ClaRe: Practical Class Incremental Learning By Remembering Previous Class Representations
• Association: Remind Your GAN not to Forget
• Collaborative Method for Incremental Learning on Classification and Generation
• Adversarial Incremental Learning
• FoCL: Feature-Oriented Continual Learning for Generative Models
• BooVAE: Boosting Approach for Continual Learning of VAE