Replay in Deep Learning: Current Approaches and Missing Biological Elements
arXiv:2104.04132
Abstract
Replay is the reactivation of one or more neural patterns, which are similar to the activation patterns experienced during past waking experiences. Replay was first observed in biological neural networks during sleep, and it is now thought to play a critical role in memory formation, retrieval, and consolidation. Replay-like mechanisms have been incorporated into deep artificial neural networks that learn over time to avoid catastrophic forgetting of previous knowledge. Replay algorithms have been successfully used in a wide range of deep learning methods within supervised, unsupervised, and reinforcement learning paradigms. In this paper, we provide the first comprehensive comparison between replay in the mammalian brain and replay in artificial neural networks. We identify multiple aspects of biological replay that are missing in deep learning systems and hypothesize how they could be utilized to improve artificial neural networks.
arXiv commentAccepted for publication in the MIT Press journal of Neural Computation
References in corpus (27)
- Overcoming catastrophic forgetting in neural networks
- Playing Atari with Deep Reinforcement Learning
- An Overview of Multi-Task Learning in Deep Neural Networks
- A continual learning survey: Defying forgetting in classification tasks
- Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence
- Continual Learning Through Synaptic Intelligence
- Cost-Effective Active Learning for Deep Image Classification
- PathNet: Evolution Channels Gradient Descent in Super Neural Networks
- Efficient Lifelong Learning with A-GEM
- Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learning
- Hindsight Experience Replay
- Which Tasks Should Be Learned Together in Multi-task Learning?
- A Comprehensive Study of Class Incremental Learning Algorithms for Visual Tasks
- Measuring Catastrophic Forgetting in Neural Networks
- Model-Free Episodic Control
- Incremental Learning for Semantic Segmentation of Large-Scale Remote Sensing Data
- Pseudo-Rehearsal: Achieving Deep Reinforcement Learning without Catastrophic Forgetting
- Neural Episodic Control
- Neurogenesis Deep Learning
- Memory Replay GANs: learning to generate images from new categories without forgetting
- Pseudo-Recursal: Solving the Catastrophic Forgetting Problem in Deep Neural Networks
- Learn the new, keep the old: Extending pretrained models with new anatomy and images
- Online Learned Continual Compression with Adaptive Quantization Modules
- Multi-Task Incremental Learning for Object Detection
- Continuous Domain Adaptation with Variational Domain-Agnostic Feature Replay
- RODEO: Replay for Online Object Detection
- Active Learning for Visual Question Answering: An Empirical Study