On Tiny Episodic Memories in Continual Learning

Abstract

In continual learning (CL), an agent learns from a stream of tasks leveraging prior experience to transfer knowledge to future tasks. It is an ideal framework to decrease the amount of supervision in the existing learning algorithms. But for a successful knowledge transfer, the learner needs to remember how to perform previous tasks. One way to endow the learner the ability to perform tasks seen in the past is to store a small memory, dubbed episodic memory, that stores few examples from previous tasks and then to replay these examples when training for future tasks. In this work, we empirically analyze the effectiveness of a very small episodic memory in a CL setup where each training example is only seen once. Surprisingly, across four rather different supervised learning benchmarks adapted to CL, a very simple baseline, that jointly trains on both examples from the current task as well as examples stored in the episodic memory, significantly outperforms specifically designed CL approaches with and without episodic memory. Interestingly, we find that repetitive training on even tiny memories of past tasks does not harm generalization, on the contrary, it improves it, with gains between 7\% and 17\% when the memory is populated with a single example per class.

Making the main point of the paper more clear

References in corpus (10)

• Distilling the Knowledge in a Neural Network
• Overcoming catastrophic forgetting in neural networks
• Playing Atari with Deep Reinforcement Learning
• Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence
• Continual Learning Through Synaptic Intelligence
• Efficient Lifelong Learning with A-GEM
• Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learning
• Experience Replay for Continual Learning
• Overcoming Catastrophic Forgetting by Incremental Moment Matching
• Lifelong Learning with Dynamically Expandable Networks

Cited by in corpus (83)

• A continual learning survey: Defying forgetting in classification tasks
• Understanding the Role of Training Regimes in Continual Learning
• Dark Experience for General Continual Learning: a Strong, Simple Baseline
• A TinyML Platform for On-Device Continual Learning with Quantized Latent Replays
• Coresets via Bilevel Optimization for Continual Learning and Streaming
• Gradient based sample selection for online continual learning
• Online Continual Learning with Maximally Interfered Retrieval
• DualNet: Continual Learning, Fast and Slow
• AdaER: An Adaptive Experience Replay Approach for Continual Lifelong Learning
• Subspace Regularizers for Few-Shot Class Incremental Learning
• Online Coreset Selection for Rehearsal-based Continual Learning
• Continual Learning for Generative Retrieval over Dynamic Corpora
• La-MAML: Look-ahead Meta Learning for Continual Learning
• An Investigation of Replay-based Approaches for Continual Learning
• CPR: Classifier-Projection Regularization for Continual Learning
• Federated Continual Learning: Concepts, Challenges, and Solutions
• Continual Learning in Low-rank Orthogonal Subspaces
• Meta-Consolidation for Continual Learning
• Meta-Learning Representations for Continual Learning
• A Dirichlet Process Mixture of Robust Task Models for Scalable Lifelong Reinforcement Learning
• Balanced softmax cross-entropy for incremental learning with and without memory
• Continual learning of longitudinal health records
• Continuous learning of spiking networks trained with local rules
• The Effectiveness of Memory Replay in Large Scale Continual Learning
• Online Lifelong Generalized Zero-Shot Learning
• New Insights on Relieving Task-Recency Bias for Online Class Incremental Learning
• Self-supervised On-device Federated Learning from Unlabeled Streams
• Carousel Memory: Rethinking the Design of Episodic Memory for Continual Learning
• Batch-level Experience Replay with Review for Continual Learning
• Linear Mode Connectivity in Multitask and Continual Learning
• Leveraging Old Knowledge to Continually Learn New Classes in Medical Images
• Tackling Catastrophic Forgetting and Background Shift in Continual Semantic Segmentation
• Enhancing Consistency and Mitigating Bias: A Data Replay Approach for Incremental Learning
• Supervised Contrastive Replay: Revisiting the Nearest Class Mean Classifier in Online Class-Incremental Continual Learning
• Generative Replay-based Continual Zero-Shot Learning
• Continual Policy Distillation of Reinforcement Learning-based Controllers for Soft Robotic In-Hand Manipulation
• Routing Networks with Co-training for Continual Learning
• Continual Learning for Monolingual End-to-End Automatic Speech Recognition
• Generalized Continual Zero-Shot Learning
• Imbalanced Continual Learning with Partitioning Reservoir Sampling
• Continual Reinforcement Learning with Multi-Timescale Replay
• Learning latent representations across multiple data domains using Lifelong VAEGAN
• Memory-Efficient Semi-Supervised Continual Learning: The World is its Own Replay Buffer
• MoP-CLIP: A Mixture of Prompt-Tuned CLIP Models for Domain Incremental Learning
• Continuous Coordination As a Realistic Scenario for Lifelong Learning
• A Study of Continual Learning Methods for Q-Learning
• Automatic Recall Machines: Internal Replay, Continual Learning and the Brain
• Multi-Domain Multi-Task Rehearsal for Lifelong Learning
• Formalizing the Generalization-Forgetting Trade-off in Continual Learning
• Continual Learning Using Multi-view Task Conditional Neural Networks
• Layerwise Optimization by Gradient Decomposition for Continual Learning
• Wide Neural Networks Forget Less Catastrophically
• Meta Continual Learning via Dynamic Programming
• Online Continual Learning with Natural Distribution Shifts: An Empirical Study with Visual Data
• Visually Grounded Continual Learning of Compositional Phrases
• Rethinking Experience Replay: a Bag of Tricks for Continual Learning
• Continual Learning via Bit-Level Information Preserving
• A Deep Learning Framework for Lifelong Machine Learning
• Block Contextual MDPs for Continual Learning
• Online Continual Learning For Visual Food Classification
• Frosting Weights for Better Continual Training
• Bilevel Continual Learning
• ADER: Adaptively Distilled Exemplar Replay Towards Continual Learning for Session-based Recommendation
• Continual Learning in the Presence of Repetition
• Continual Learning: Forget-free Winning Subnetworks for Video Representations
• Navigating Out-of-Distribution Electricity Load Forecasting during COVID-19: Benchmarking energy load forecasting models without and with continual learning
• Graph-Based Continual Learning
• A Conceptual Framework for Lifelong Learning
• Learning Neural Models for Natural Language Processing in the Face of Distributional Shift
• Unsupervised Continual Learning Via Pseudo Labels
• Personalizing Pre-trained Models
• How do Quadratic Regularizers Prevent Catastrophic Forgetting: The Role of Interpolation
• An EM Framework for Online Incremental Learning of Semantic Segmentation
• Catastrophic Forgetting in Deep Graph Networks: an Introductory Benchmark for Graph Classification
• Learning to Continually Learn Rapidly from Few and Noisy Data
• iLGaCo: Incremental Learning of Gait Covariate Factors
• TAG: Task-based Accumulated Gradients for Lifelong learning
• CALM: Continuous Adaptive Learning for Language Modeling
• Online Continual Learning in Image Classification: An Empirical Survey
• Dynamic VAEs with Generative Replay for Continual Zero-shot Learning
• Lifelong Mixture of Variational Autoencoders
• Shared and Private VAEs with Generative Replay for Continual Learning
• Lifelong Twin Generative Adversarial Networks