An Investigation of Replay-based Approaches for Continual Learning

Abstract

Continual learning (CL) is a major challenge of machine learning (ML) and describes the ability to learn several tasks sequentially without catastrophic forgetting (CF). Recent works indicate that CL is a complex topic, even more so when real-world scenarios with multiple constraints are involved. Several solution classes have been proposed, of which so-called replay-based approaches seem very promising due to their simplicity and robustness. Such approaches store a subset of past samples in a dedicated memory for later processing: while this does not solve all problems, good results have been obtained. In this article, we empirically investigate replay-based approaches of continual learning and assess their potential for applications. Selected recent approaches as well as own proposals are compared on a common set of benchmarks, with a particular focus on assessing the performance of different sample selection strategies. We find that the impact of sample selection increases when a smaller number of samples is stored. Nevertheless, performance varies strongly between different replay approaches. Surprisingly, we find that the most naive rehearsal-based approaches that we propose here can outperform recent state-of-the-art methods.

arXiv commentAccepted at the IJCNN2021, 9 pages, 1 figure

References in corpus (22)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Overcoming catastrophic forgetting in neural networks
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• A continual learning survey: Defying forgetting in classification tasks
• Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence
• Continual Learning Through Synaptic Intelligence
• PathNet: Evolution Channels Gradient Descent in Super Neural Networks
• Efficient Lifelong Learning with A-GEM
• Gradient Episodic Memory for Continual Learning
• On Tiny Episodic Memories in Continual Learning
• Overcoming Catastrophic Forgetting by Incremental Moment Matching
• Mnemonics Training: Multi-Class Incremental Learning without Forgetting
• Overcoming catastrophic forgetting with hard attention to the task
• Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselines
• Measuring Catastrophic Forgetting in Neural Networks
• Generative replay with feedback connections as a general strategy for continual learning
• Deep Generative Dual Memory Network for Continual Learning
• Born to Learn: the Inspiration, Progress, and Future of Evolved Plastic Artificial Neural Networks
• A comprehensive, application-oriented study of catastrophic forgetting in DNNs
• Selfless Sequential Learning
• Optimal Continual Learning has Perfect Memory and is NP-hard
• Training Discriminative Models to Evaluate Generative Ones

Cited by in corpus (2)

• A Study of Continual Learning Methods for Q-Learning
• Expanding continual few-shot learning benchmarks to include recognition of specific instances