Efficient Lifelong Learning with A-GEM

Abstract

In lifelong learning, the learner is presented with a sequence of tasks, incrementally building a data-driven prior which may be leveraged to speed up learning of a new task. In this work, we investigate the efficiency of current lifelong approaches, in terms of sample complexity, computational and memory cost. Towards this end, we first introduce a new and a more realistic evaluation protocol, whereby learners observe each example only once and hyper-parameter selection is done on a small and disjoint set of tasks, which is not used for the actual learning experience and evaluation. Second, we introduce a new metric measuring how quickly a learner acquires a new skill. Third, we propose an improved version of GEM (Lopez-Paz & Ranzato, 2017), dubbed Averaged GEM (A-GEM), which enjoys the same or even better performance as GEM, while being almost as computationally and memory efficient as EWC (Kirkpatrick et al., 2016) and other regularization-based methods. Finally, we show that all algorithms including A-GEM can learn even more quickly if they are provided with task descriptors specifying the classification tasks under consideration. Our experiments on several standard lifelong learning benchmarks demonstrate that A-GEM has the best trade-off between accuracy and efficiency.

Published as a conference paper at ICLR 2019

Cited by in corpus (149)

• A continual learning survey: Defying forgetting in classification tasks
• On Tiny Episodic Memories in Continual Learning
• Towards Robust Evaluations of Continual Learning
• Continual Learning for Recurrent Neural Networks: an Empirical Evaluation
• LAMOL: LAnguage MOdeling for Lifelong Language Learning
• Lifelong Generative Modeling
• Gradient Surgery for Multi-Task Learning
• Incremental Object Detection via Meta-Learning
• A Neural Dirichlet Process Mixture Model for Task-Free Continual Learning
• Dark Experience for General Continual Learning: a Strong, Simple Baseline
• Online Continual Learning with Maximally Interfered Retrieval
• Gradient based sample selection for online continual learning
• Continual Learning in Sensor-based Human Activity Recognition: an Empirical Benchmark Analysis
• DualNet: Continual Learning, Fast and Slow
• Continual Deep Learning by Functional Regularisation of Memorable Past
• Overcoming Long-term Catastrophic Forgetting through Adversarial Neural Pruning and Synaptic Consolidation
• Triple Memory Networks: a Brain-Inspired Method for Continual Learning
• Efficient Continual Learning in Neural Networks with Embedding Regularization
• Continual Learning and Catastrophic Forgetting
• M2KD: Multi-model and Multi-level Knowledge Distillation for Incremental Learning
• Online Coreset Selection for Rehearsal-based Continual Learning
• Lifelong Continual Learning for Anomaly Detection: New Challenges, Perspectives, and Insights
• La-MAML: Look-ahead Meta Learning for Continual Learning
• LFPT5: A Unified Framework for Lifelong Few-shot Language Learning Based on Prompt Tuning of T5
• Class-incremental Learning with Pre-allocated Fixed Classifiers
• An Investigation of Replay-based Approaches for Continual Learning
• Federated Continual Learning with Weighted Inter-client Transfer
• Continual Learning in Low-rank Orthogonal Subspaces
• Efficient Continual Learning with Modular Networks and Task-Driven Priors
• Class-incremental Learning via Deep Model Consolidation
• Meta-Consolidation for Continual Learning
• Improved Schemes for Episodic Memory-based Lifelong Learning
• Gradient Projection Memory for Continual Learning
• Uncertainty-guided Continual Learning with Bayesian Neural Networks
• Meta-Learning Representations for Continual Learning
• Continual Learning via Inter-Task Synaptic Mapping
• Self-Supervised Training Enhances Online Continual Learning
• AdaptCL: Adaptive Continual Learning for Tackling Heterogeneity in Sequential Datasets
• Replay in Deep Learning: Current Approaches and Missing Biological Elements
• REMIND Your Neural Network to Prevent Catastrophic Forgetting
• Continual learning of longitudinal health records
• Continual World: A Robotic Benchmark For Continual Reinforcement Learning
• Progressive Memory Banks for Incremental Domain Adaptation
• The Effectiveness of Memory Replay in Large Scale Continual Learning
• New Insights on Relieving Task-Recency Bias for Online Class Incremental Learning
• Scalable and Order-robust Continual Learning with Additive Parameter Decomposition
• Continual Adaptation of Semantic Segmentation using Complementary 2D-3D Data Representations
• Objects in Semantic Topology
• Batch-level Experience Replay with Review for Continual Learning
• Language-Inspired Relation Transfer for Few-shot Class-Incremental Learning
• Carousel Memory: Rethinking the Design of Episodic Memory for Continual Learning
• A Theoretical Analysis of Catastrophic Forgetting through the NTK Overlap Matrix
• Meta-Learning with Sparse Experience Replay for Lifelong Language Learning
• Class Gradient Projection For Continual Learning
• Online Class-Incremental Continual Learning with Adversarial Shapley Value
• Online Fast Adaptation and Knowledge Accumulation: a New Approach to Continual Learning
• Continual Learning in Neural Networks
• Lifelong Machine Learning with Deep Streaming Linear Discriminant Analysis
• Linear Mode Connectivity in Multitask and Continual Learning
• Generalisation Guarantees for Continual Learning with Orthogonal Gradient Descent
• Tackling Catastrophic Forgetting and Background Shift in Continual Semantic Segmentation
• Generative Replay-based Continual Zero-Shot Learning
• Continual Learning for Monolingual End-to-End Automatic Speech Recognition
• Supervised Contrastive Replay: Revisiting the Nearest Class Mean Classifier in Online Class-Incremental Continual Learning
• Neural Architecture Search for Class-incremental Learning
• Generative Feature Replay For Class-Incremental Learning
• Compositional Generalization for Primitive Substitutions
• Imbalanced Continual Learning with Partitioning Reservoir Sampling
• Gradient Regularized Contrastive Learning for Continual Domain Adaptation
• Supermasks in Superposition
• Hypernetworks for Continual Semi-Supervised Learning
• Online Structured Meta-learning
• Evaluating Online Continual Learning with CALM
• Memory-Efficient Semi-Supervised Continual Learning: The World is its Own Replay Buffer
• Continuous Coordination As a Realistic Scenario for Lifelong Learning
• Automatic Recall Machines: Internal Replay, Continual Learning and the Brain
• Beyond Imitation: A Life-long Policy Learning Framework for Path Tracking Control of Autonomous Driving
• A Study of Continual Learning Methods for Q-Learning
• Lifelong Learning of Hate Speech Classification on Social Media
• On the importance of cross-task features for class-incremental learning
• Understanding Continual Learning Settings with Data Distribution Drift Analysis
• Rethinking Curriculum Learning with Incremental Labels and Adaptive Compensation
• Pseudo-Rehearsal for Continual Learning with Normalizing Flows
• Online Continual Learning on Class Incremental Blurry Task Configuration with Anytime Inference
• CUCL: Codebook for Unsupervised Continual Learning
• Multi-Domain Multi-Task Rehearsal for Lifelong Learning
• CrowdTransfer: Enabling Crowd Knowledge Transfer in AIoT Community
• DRILL: Dynamic Representations for Imbalanced Lifelong Learning
• Continual Learning for Text Classification with Information Disentanglement Based Regularization
• Look At Me, No Replay! SurpriseNet: Anomaly Detection Inspired Class Incremental Learning
• Sequoia: A Software Framework to Unify Continual Learning Research
• Continual Learning with Fully Probabilistic Models
• Towards Better Plasticity-Stability Trade-off in Incremental Learning: A Simple Linear Connector
• Wide Neural Networks Forget Less Catastrophically
• Visually Grounded Continual Learning of Compositional Phrases
• Meta-Learned Attribute Self-Gating for Continual Generalized Zero-Shot Learning
• Neuromodulated Neural Architectures with Local Error Signals for Memory-Constrained Online Continual Learning
• Efficient Meta Lifelong-Learning with Limited Memory
• Continual Learning: Tackling Catastrophic Forgetting in Deep Neural Networks with Replay Processes
• Bilevel Continual Learning
• Reviewing continual learning from the perspective of human-level intelligence
• Memory Efficient Class-Incremental Learning for Image Classification
• Optimizing Reusable Knowledge for Continual Learning via Metalearning
• OER: Offline Experience Replay for Continual Offline Reinforcement Learning
• Rethinking Experience Replay: a Bag of Tricks for Continual Learning
• Continual Learning via Bit-Level Information Preserving
• Frosting Weights for Better Continual Training
• A Conceptual Framework for Lifelong Learning
• Disentangle-based Continual Graph Representation Learning
• Continual Learning in Task-Oriented Dialogue Systems
• Curriculum-Meta Learning for Order-Robust Continual Relation Extraction
• Incremental Meta-Learning via Indirect Discriminant Alignment
• CLEVA-Compass: A Continual Learning EValuation Assessment Compass to Promote Research Transparency and Comparability
• Life-Long Multi-Task Learning of Adaptive Path Tracking Policy for Autonomous Vehicle
• Lifelong Learning for Neural powered Mixed Integer Programming
• Graph-Based Continual Learning
• Learning Continually from Low-shot Data Stream
• Fixed Points in Cyber Space: Rethinking Optimal Evasion Attacks in the Age of AI-NIDS
• CLOPS: Continual Learning of Physiological Signals
• TIE: A Framework for Embedding-based Incremental Temporal Knowledge Graph Completion
• Tuned Compositional Feature Replays for Efficient Stream Learning
• Dynamic Continual Learning: Harnessing Parameter Uncertainty for Improved Network Adaptation
• Continual Speaker Adaptation for Text-to-Speech Synthesis
• An EM Framework for Online Incremental Learning of Semantic Segmentation
• How do Quadratic Regularizers Prevent Catastrophic Forgetting: The Role of Interpolation
• Continual Machine Reading Comprehension via Uncertainty-aware Fixed Memory and Adversarial Domain Adaptation
• Visually Grounded Continual Language Learning with Selective Specialization
• Catastrophic Forgetting in Deep Graph Networks: an Introductory Benchmark for Graph Classification
• Selective Replay Enhances Learning in Online Continual Analogical Reasoning
• DeCoR: Defy Knowledge Forgetting by Predicting Earlier Audio Codes
• Lifelong Learning based Disease Diagnosis on Clinical Notes
• Learning Representations on the Unit Sphere: Investigating Angular Gaussian and von Mises-Fisher Distributions for Online Continual Learning
• Defeating Catastrophic Forgetting via Enhanced Orthogonal Weights Modification
• Learning to Continually Learn Rapidly from Few and Noisy Data
• Continually Learn to Map Visual Concepts to Large Language Models in Resource-constrained Environments
• Kronecker Factorization for Preventing Catastrophic Forgetting in Large-scale Medical Entity Linking
• Towards Continual Entity Learning in Language Models for Conversational Agents
• MyMigrationBot: A Cloud-based Facebook Social Chatbot for Migrant Populations
• Weight Friction: A Simple Method to Overcome Catastrophic Forgetting and Enable Continual Learning
• TAG: Task-based Accumulated Gradients for Lifelong learning
• Piggyback GAN: Efficient Lifelong Learning for Image Conditioned Generation
• Online Continual Learning in Image Classification: An Empirical Survey
• Lifelong Mixture of Variational Autoencoders
• Schematic Memory Persistence and Transience for Efficient and Robust Continual Learning
• Compression-aware Continual Learning using Singular Value Decomposition
• Class-incremental Learning with Rectified Feature-Graph Preservation
• Lifelong Twin Generative Adversarial Networks
• Continual Learning of Multi-modal Dynamics with External Memory
• Incremental Cross-Domain Adaptation for Robust Retinopathy Screening via Bayesian Deep Learning