Continual Learning with Deep Generative Replay

Abstract

Attempts to train a comprehensive artificial intelligence capable of solving multiple tasks have been impeded by a chronic problem called catastrophic forgetting. Although simply replaying all previous data alleviates the problem, it requires large memory and even worse, often infeasible in real world applications where the access to past data is limited. Inspired by the generative nature of hippocampus as a short-term memory system in primate brain, we propose the Deep Generative Replay, a novel framework with a cooperative dual model architecture consisting of a deep generative model ("generator") and a task solving model ("solver"). With only these two models, training data for previous tasks can easily be sampled and interleaved with those for a new task. We test our methods in several sequential learning settings involving image classification tasks.

NIPS 2017

References in corpus (4)

• Overcoming catastrophic forgetting in neural networks
• Improved Training of Wasserstein GANs
• An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks
• Online Contrastive Divergence with Generative Replay: Experience Replay without Storing Data

Cited by in corpus (80)

• Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence
• Three scenarios for continual learning
• Born Again Neural Networks
• How Generative Adversarial Networks and Their Variants Work: An Overview
• A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
• Overcoming catastrophic forgetting with hard attention to the task
• A Comprehensive Study of Class Incremental Learning Algorithms for Visual Tasks
• Deep Generative Dual Memory Network for Continual Learning
• LAMOL: LAnguage MOdeling for Lifelong Language Learning
• Incremental Object Detection via Meta-Learning
• Understanding the Role of Training Regimes in Continual Learning
• Functional Regularisation for Continual Learning with Gaussian Processes
• Gradient based sample selection for online continual learning
• Task Agnostic Continual Learning Using Online Variational Bayes
• Continual Learning in Sensor-based Human Activity Recognition: an Empirical Benchmark Analysis
• Deep Cocktail Network: Multi-source Unsupervised Domain Adaptation with Category Shift
• Learning to Answer by Learning to Ask: Getting the Best of GPT-2 and BERT Worlds
• CLeaR: An Adaptive Continual Learning Framework for Regression Tasks
• Learning offline: memory replay in biological and artificial reinforcement learning
• Rotate your Networks: Better Weight Consolidation and Less Catastrophic Forgetting
• Continual Learning for Fake Audio Detection
• Maintaining Discrimination and Fairness in Class Incremental Learning
• Continual Learning with Node-Importance based Adaptive Group Sparse Regularization
• Gradient Projection Memory for Continual Learning
• SupportNet: solving catastrophic forgetting in class incremental learning with support data
• Meta Continual Learning
• RATT: Recurrent Attention to Transient Tasks for Continual Image Captioning
• Continual Learning via Inter-Task Synaptic Mapping
• Flattening Sharpness for Dynamic Gradient Projection Memory Benefits Continual Learning
• Progressive Reinforcement Learning with Distillation for Multi-Skilled Motion Control
• Continual Reinforcement Learning with Complex Synapses
• Machine Learning for the Detection and Identification of Internet of Things (IoT) Devices: A Survey
• AirLoop: Lifelong Loop Closure Detection
• Online Lifelong Generalized Zero-Shot Learning
• A First Look at Class Incremental Learning in Deep Learning Mobile Traffic Classification
• ACE: Adapting to Changing Environments for Semantic Segmentation
• Ecological Reinforcement Learning
• Rethinking Architecture Design for Tackling Data Heterogeneity in Federated Learning
• Lifelong Learning of Compositional Structures
• Keep and Learn: Continual Learning by Constraining the Latent Space for Knowledge Preservation in Neural Networks
• A review of mobile robot motion planning methods: from classical motion planning workflows to reinforcement learning-based architectures
• Gradient Episodic Memory with a Soft Constraint for Continual Learning
• Learning latent representations across multiple data domains using Lifelong VAEGAN
• Learning from the Past: Continual Meta-Learning via Bayesian Graph Modeling
• XtarNet: Learning to Extract Task-Adaptive Representation for Incremental Few-Shot Learning
• On The State of Data In Computer Vision: Human Annotations Remain Indispensable for Developing Deep Learning Models
• Powerpropagation: A sparsity inducing weight reparameterisation
• On the importance of cross-task features for class-incremental learning
• NIPS - Not Even Wrong? A Systematic Review of Empirically Complete Demonstrations of Algorithmic Effectiveness in the Machine Learning and Artificial Intelligence Literature
• Re-using Adversarial Mask Discriminators for Test-time Training under Distribution Shifts
• Lifelong Learning with Sketched Structural Regularization
• Doubly Nested Network for Resource-Efficient Inference
• Lifelong Vehicle Trajectory Prediction Framework Based on Generative Replay
• Disjoint Multi-task Learning between Heterogeneous Human-centric Tasks
• Continual Learning via Bit-Level Information Preserving
• Less-forgetful Learning for Domain Expansion in Deep Neural Networks
• Memory-Free Generative Replay For Class-Incremental Learning
• GROWN: GRow Only When Necessary for Continual Learning
• Disentanglement of Color and Shape Representations for Continual Learning
• Incremental Embedding Learning via Zero-Shot Translation
• Variational Auto-Regressive Gaussian Processes for Continual Learning
• How good is my GAN?
• Bilevel Continual Learning
• When Video Classification Meets Incremental Classes
• Unsupervised Continual Learning Via Pseudo Labels
• Unsupervised Class-Incremental Learning Through Confusion
• Self-Net: Lifelong Learning via Continual Self-Modeling
• DeGAN : Data-Enriching GAN for Retrieving Representative Samples from a Trained Classifier
• Adversarial Incremental Learning
• ClaRe: Practical Class Incremental Learning By Remembering Previous Class Representations
• Collaborative Method for Incremental Learning on Classification and Generation
• Center Loss Regularization for Continual Learning
• Dynamic VAEs with Generative Replay for Continual Zero-shot Learning
• RSAC: Regularized Subspace Approximation Classifier for Lightweight Continuous Learning
• Discriminative Distillation to Reduce Class Confusion in Continual Learning
• SeNA-CNN: Overcoming Catastrophic Forgetting in Convolutional Neural Networks by Selective Network Augmentation
• Towards Heterogeneous Clients with Elastic Federated Learning
• Continual learning using hash-routed convolutional neural networks
• Shared and Private VAEs with Generative Replay for Continual Learning
• ZS-IL: Looking Back on Learned Experiences For Zero-Shot Incremental Learning