Three scenarios for continual learning

Abstract

Standard artificial neural networks suffer from the well-known issue of catastrophic forgetting, making continual or lifelong learning difficult for machine learning. In recent years, numerous methods have been proposed for continual learning, but due to differences in evaluation protocols it is difficult to directly compare their performance. To enable more structured comparisons, we describe three continual learning scenarios based on whether at test time task identity is provided and--in case it is not--whether it must be inferred. Any sequence of well-defined tasks can be performed according to each scenario. Using the split and permuted MNIST task protocols, for each scenario we carry out an extensive comparison of recently proposed continual learning methods. We demonstrate substantial differences between the three scenarios in terms of difficulty and in terms of how efficient different methods are. In particular, when task identity must be inferred (i.e., class incremental learning), we find that regularization-based approaches (e.g., elastic weight consolidation) fail and that replaying representations of previous experiences seems required for solving this scenario.

Extended version of work presented at the NeurIPS Continual Learning workshop (2018); 18 pages, 5 figures, 6 tables. Related to arXiv:1809.10635

References in corpus (19)

• Adam: A Method for Stochastic Optimization
• Distilling the Knowledge in a Neural Network
• Tutorial on Variational Autoencoders
• Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence
• Continual Learning Through Synaptic Intelligence
• An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks
• PathNet: Evolution Channels Gradient Descent in Super Neural Networks
• New insights and perspectives on the natural gradient method
• Variational Continual Learning
• Overcoming catastrophic forgetting with hard attention to the task
• Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselines
• Continual Learning with Deep Generative Replay
• Generative replay with feedback connections as a general strategy for continual learning
• Towards Robust Evaluations of Continual Learning
• Deep Generative Dual Memory Network for Continual Learning
• On Quadratic Penalties in Elastic Weight Consolidation
• Incremental Classifier Learning with Generative Adversarial Networks
• Task Agnostic Continual Learning Using Online Variational Bayes
• A Strategy for an Uncompromising Incremental Learner

Cited by in corpus (108)

• Generative replay with feedback connections as a general strategy for continual learning
• Continual Learning for Recurrent Neural Networks: an Empirical Evaluation
• Deep Transfer Learning for Industrial Automation: A Review and Discussion of New Techniques for Data-Driven Machine Learning
• Federated Continual Learning via Knowledge Fusion: A Survey
• A TinyML Platform for On-Device Continual Learning with Quantized Latent Replays
• Concept drift detection and adaptation for federated and continual learning
• Continual Learning in Sensor-based Human Activity Recognition: an Empirical Benchmark Analysis
• SpaceNet: Make Free Space For Continual Learning
• Lifelong Teacher-Student Network Learning
• Domain-Incremental Continual Learning for Mitigating Bias in Facial Expression and Action Unit Recognition
• Avoiding Catastrophe: Active Dendrites Enable Multi-Task Learning in Dynamic Environments
• Triple Memory Networks: a Brain-Inspired Method for Continual Learning
• Continual Learning of a Mixed Sequence of Similar and Dissimilar Tasks
• AdaER: An Adaptive Experience Replay Approach for Continual Lifelong Learning
• Lifelong Continual Learning for Anomaly Detection: New Challenges, Perspectives, and Insights
• Class-incremental Learning with Pre-allocated Fixed Classifiers
• Maintaining Discrimination and Fairness in Class Incremental Learning
• Federated Continual Learning: Concepts, Challenges, and Solutions
• Efficient Continual Learning with Modular Networks and Task-Driven Priors
• Meta-Consolidation for Continual Learning
• Continual Learning via Inter-Task Synaptic Mapping
• ECG-CL: A Comprehensive Electrocardiogram Interpretation Method Based on Continual Learning
• Continual learning of longitudinal health records
• Elastic Weight Consolidation (EWC): Nuts and Bolts
• Learning Representations for New Sound Classes With Continual Self-Supervised Learning
• The Effectiveness of Memory Replay in Large Scale Continual Learning
• New Insights on Relieving Task-Recency Bias for Online Class Incremental Learning
• A Domain-Agnostic Approach for Characterization of Lifelong Learning Systems
• Adaptive Masked Proxies for Few-Shot Segmentation
• Semantic Drift Compensation for Class-Incremental Learning
• A First Look at Class Incremental Learning in Deep Learning Mobile Traffic Classification
• A Neural Span-Based Continual Named Entity Recognition Model
• Continual Learning on the Edge with TensorFlow Lite
• Online Fast Adaptation and Knowledge Accumulation: a New Approach to Continual Learning
• Continual Learning in Recurrent Neural Networks
• FeTrIL: Feature Translation for Exemplar-Free Class-Incremental Learning
• Exploring UMAP in hybrid models of entropy-based and representativeness sampling for active learning in biomedical segmentation
• Generalisation Guarantees for Continual Learning with Orthogonal Gradient Descent
• Encoders and Ensembles for Task-Free Continual Learning
• Hierarchical Indian Buffet Neural Networks for Bayesian Continual Learning
• Generative Feature Replay For Class-Incremental Learning
• Continual Policy Distillation of Reinforcement Learning-based Controllers for Soft Robotic In-Hand Manipulation
• Towards lifelong learning of Recurrent Neural Networks for control design
• Privacy-preserving Continual Federated Clustering via Adaptive Resonance Theory
• Imbalanced Continual Learning with Partitioning Reservoir Sampling
• Adaptive Online Planning for Continual Lifelong Learning
• Continual Learning with Neuromorphic Computing: Foundations, Methods, and Emerging Applications
• Supermasks in Superposition
• Self-Supervised Learning Aided Class-Incremental Lifelong Learning
• Continual Learning using a Bayesian Nonparametric Dictionary of Weight Factors
• Memory-Efficient Semi-Supervised Continual Learning: The World is its Own Replay Buffer
• Powerpropagation: A sparsity inducing weight reparameterisation
• Digital Twin Backed Closed-Loops for Energy-Aware and Open RAN-based Fixed Wireless Access Serving Rural Areas
• Towards Fair Affective Robotics: Continual Learning for Mitigating Bias in Facial Expression and Action Unit Recognition
• Understanding Continual Learning Settings with Data Distribution Drift Analysis
• Automatic Recall Machines: Internal Replay, Continual Learning and the Brain
• Continual Model-Based Reinforcement Learning with Hypernetworks
• Energy-Based Models for Continual Learning
• Unifying Regularisation Methods for Continual Learning
• Posterior Meta-Replay for Continual Learning
• Lifelong Learning using Eigentasks: Task Separation, Skill Acquisition, and Selective Transfer
• Graceful Degradation and Related Fields
• Sequoia: A Software Framework to Unify Continual Learning Research
• Continual Learning Using Multi-view Task Conditional Neural Networks
• Learning Invariant Representation for Continual Learning
• Layerwise Optimization by Gradient Decomposition for Continual Learning
• Look At Me, No Replay! SurpriseNet: Anomaly Detection Inspired Class Incremental Learning
• CoReS: Compatible Representations via Stationarity
• Task-agnostic Continual Learning with Hybrid Probabilistic Models
• Reviewing continual learning from the perspective of human-level intelligence
• Optimizing Reusable Knowledge for Continual Learning via Metalearning
• GAN Memory with No Forgetting
• CAM-GAN: Continual Adaptation Modules for Generative Adversarial Networks
• RECALL: Rehearsal-free Continual Learning for Object Classification
• CatNet: Class Incremental 3D ConvNets for Lifelong Egocentric Gesture Recognition
• Human Inspired Progressive Alignment and Comparative Learning for Grounded Word Acquisition
• Unveiling the Tapestry: the Interplay of Generalization and Forgetting in Continual Learning
• CLEVA-Compass: A Continual Learning EValuation Assessment Compass to Promote Research Transparency and Comparability
• Learn Faster and Forget Slower via Fast and Stable Task Adaptation
• Variational Auto-Regressive Gaussian Processes for Continual Learning
• CLOPS: Continual Learning of Physiological Signals
• Federated Reconnaissance: Efficient, Distributed, Class-Incremental Learning
• Visually Grounded Continual Language Learning with Selective Specialization
• Incremental Learning via Rate Reduction
• Regularization-based Continual Learning for Fault Prediction in Lithium-Ion Batteries
• Generative Feature Replay with Orthogonal Weight Modification for Continual Learning
• Energy Aligning for Biased Models
• Continual Learning in Deep Networks: an Analysis of the Last Layer
• Better Knowledge Retention through Metric Learning
• Center Loss Regularization for Continual Learning
• Self-Promoted Prototype Refinement for Few-Shot Class-Incremental Learning
• Target Layer Regularization for Continual Learning Using Cramer-Wold Generator
• Defeating Catastrophic Forgetting via Enhanced Orthogonal Weights Modification
• Principal Gradient Direction and Confidence Reservoir Sampling for Continual Learning
• Continually Learn to Map Visual Concepts to Large Language Models in Resource-constrained Environments
• Condensed Composite Memory Continual Learning
• Toward industrial use of continual learning : new metrics proposal for class incremental learning
• Weighted Ensemble Models Are Strong Continual Learners
• Lifelong Learning Without a Task Oracle
• Dendritic Self-Organizing Maps for Continual Learning
• TAG: Task-based Accumulated Gradients for Lifelong learning
• Structured Ensembles: an Approach to Reduce the Memory Footprint of Ensemble Methods
• OvA-INN: Continual Learning with Invertible Neural Networks
• Online Continual Learning in Image Classification: An Empirical Survey
• Split-and-Bridge: Adaptable Class Incremental Learning within a Single Neural Network
• Continual Competitive Memory: A Neural System for Online Task-Free Lifelong Learning
• Lifelong Mixture of Variational Autoencoders
• Disentangling Transfer and Interference in Multi-Domain Learning