Continual Learning and Catastrophic Forgetting
arXiv:2403.05175 · doi:10.1016/B978-0-443-15754-7.00073-0
Abstract
This book chapter delves into the dynamics of continual learning, which is the process of incrementally learning from a non-stationary stream of data. Although continual learning is a natural skill for the human brain, it is very challenging for artificial neural networks. An important reason is that, when learning something new, these networks tend to quickly and drastically forget what they had learned before, a phenomenon known as catastrophic forgetting. Especially in the last decade, continual learning has become an extensively studied topic in deep learning. This book chapter reviews the insights that this field has generated.
Preprint of a book chapter; 21 pages, 4 figures
References in corpus (17)
- Efficient Lifelong Learning with A-GEM
- A Comprehensive Study of Class Incremental Learning Algorithms for Visual Tasks
- Compacting, Picking and Growing for Unforgetting Continual Learning
- A Neural Dirichlet Process Mixture Model for Task-Free Continual Learning
- Continual Learning via Neural Pruning
- Understanding the Role of Training Regimes in Continual Learning
- Incorporating Neuro-Inspired Adaptability for Continual Learning in Artificial Intelligence
- Learning Fast, Learning Slow: A General Continual Learning Method based on Complementary Learning System
- Flattening Sharpness for Dynamic Gradient Projection Memory Benefits Continual Learning
- A Theoretical Study on Solving Continual Learning
- Continual Learning: Applications and the Road Forward
- Continual Learning with Foundation Models: An Empirical Study of Latent Replay
- BiRT: Bio-inspired Replay in Vision Transformers for Continual Learning
- Continual Learning via Sequential Function-Space Variational Inference
- TACOS: Task Agnostic Continual Learning in Spiking Neural Networks
- Continual Learning in the Presence of Repetition
- Two Complementary Perspectives to Continual Learning: Ask Not Only What to Optimize, But Also How