Generative replay with feedback connections as a general strategy for continual learning

Abstract

A major obstacle to developing artificial intelligence applications capable of true lifelong learning is that artificial neural networks quickly or catastrophically forget previously learned tasks when trained on a new one. Numerous methods for alleviating catastrophic forgetting are currently being proposed, but differences in evaluation protocols make it difficult to directly compare their performance. To enable more meaningful comparisons, here we identified three distinct scenarios for continual learning based on whether task identity is known and, if it is not, whether it needs to be inferred. Performing the split and permuted MNIST task protocols according to each of these scenarios, we found that regularization-based approaches (e.g., elastic weight consolidation) failed when task identity needed to be inferred. In contrast, generative replay combined with distillation (i.e., using class probabilities as "soft targets") achieved superior performance in all three scenarios. Addressing the issue of efficiency, we reduced the computational cost of generative replay by integrating the generative model into the main model by equipping it with generative feedback or backward connections. This Replay-through-Feedback approach substantially shortened training time with no or negligible loss in performance. We believe this to be an important first step towards making the powerful technique of generative replay scalable to real-world continual learning applications.

arXiv comment17 pages, 8 figures, 4 tables

References in corpus (20)

• Adam: A Method for Stochastic Optimization
• Distilling the Knowledge in a Neural Network
• Overcoming catastrophic forgetting in neural networks
• Pixel Recurrent Neural Networks
• Tutorial on Variational Autoencoders
• Continual Learning Through Synaptic Intelligence
• An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks
• PathNet: Evolution Channels Gradient Descent in Super Neural Networks
• Three scenarios for continual learning
• FearNet: Brain-Inspired Model for Incremental Learning
• New insights and perspectives on the natural gradient method
• Variational Continual Learning
• Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilization
• Overcoming catastrophic forgetting with hard attention to the task
• Towards Robust Evaluations of Continual Learning
• Deep Generative Dual Memory Network for Continual Learning
• On Quadratic Penalties in Elastic Weight Consolidation
• Incremental Classifier Learning with Generative Adversarial Networks
• Pseudo-Recursal: Solving the Catastrophic Forgetting Problem in Deep Neural Networks
• A Strategy for an Uncompromising Incremental Learner

Cited by in corpus (59)

• Three scenarios for continual learning
• Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselines
• Continual Learning for Recurrent Neural Networks: an Empirical Evaluation
• Task Agnostic Continual Learning Using Online Variational Bayes
• DualNet: Continual Learning, Fast and Slow
• CLeaR: An Adaptive Continual Learning Framework for Regression Tasks
• Recent Advances of Continual Learning in Computer Vision: An Overview
• An Investigation of Replay-based Approaches for Continual Learning
• Improving and Understanding Variational Continual Learning
• Continual Learning with Adaptive Weights (CLAW)
• Meta-Consolidation for Continual Learning
• Continual Learning via Inter-Task Synaptic Mapping
• Lifelong Object Detection
• When Meta-Learning Meets Online and Continual Learning: A Survey
• A Domain-Agnostic Approach for Characterization of Lifelong Learning Systems
• An Adaptive Random Path Selection Approach for Incremental Learning
• Continual Learning in Recurrent Neural Networks
• Generalisation Guarantees for Continual Learning with Orthogonal Gradient Descent
• Variational Prototype Replays for Continual Learning
• Neural Architecture Search for Class-incremental Learning
• On the Exploration of Incremental Learning for Fine-grained Image Retrieval
• Continual Learning using a Bayesian Nonparametric Dictionary of Weight Factors
• If deep learning is the answer, then what is the question?
• Powerpropagation: A sparsity inducing weight reparameterisation
• Automatic Recall Machines: Internal Replay, Continual Learning and the Brain
• Lifelong Neural Predictive Coding: Learning Cumulatively Online without Forgetting
• OpenLORIS-Object: A Robotic Vision Dataset and Benchmark for Lifelong Deep Learning
• Localizing Catastrophic Forgetting in Neural Networks
• Product Kanerva Machines: Factorized Bayesian Memory
• Lifelong Language Knowledge Distillation
• Multi-Domain Multi-Task Rehearsal for Lifelong Learning
• Formalizing the Generalization-Forgetting Trade-off in Continual Learning
• Lifelong Learning using Eigentasks: Task Separation, Skill Acquisition, and Selective Transfer
• A Combinatorial Perspective on Transfer Learning
• Posterior Meta-Replay for Continual Learning
• Lifelong Vehicle Trajectory Prediction Framework Based on Generative Replay
• Meta-Learned Attribute Self-Gating for Continual Generalized Zero-Shot Learning
• Reviewing continual learning from the perspective of human-level intelligence
• Learn-Prune-Share for Lifelong Learning
• CAM-GAN: Continual Adaptation Modules for Generative Adversarial Networks
• GAN Memory with No Forgetting
• Generative Memory for Lifelong Reinforcement Learning
• Label Mapping Neural Networks with Response Consolidation for Class Incremental Learning
• Continual Learning via Bit-Level Information Preserving
• Memory Efficient Class-Incremental Learning for Image Classification
• Neuromodulated Neural Architectures with Local Error Signals for Memory-Constrained Online Continual Learning
• IROS 2019 Lifelong Robotic Vision Challenge -- Lifelong Object Recognition Report
• Learning to Remember from a Multi-Task Teacher
• One-Shot Domain Incremental Learning
• Incremental Concept Learning via Online Generative Memory Recall
• Mixture-of-Variational-Experts for Continual Learning
• Online Continual Learning in Image Classification: An Empirical Survey
• Center Loss Regularization for Continual Learning
• Continual learning using hash-routed convolutional neural networks
• Better Knowledge Retention through Metric Learning
• Condensed Composite Memory Continual Learning
• Do Not Forget to Attend to Uncertainty while Mitigating Catastrophic Forgetting
• FoCL: Feature-Oriented Continual Learning for Generative Models
• RSAC: Regularized Subspace Approximation Classifier for Lightweight Continuous Learning