Replay in Deep Learning: Current Approaches and Missing Biological Elements

Abstract

Replay is the reactivation of one or more neural patterns, which are similar to the activation patterns experienced during past waking experiences. Replay was first observed in biological neural networks during sleep, and it is now thought to play a critical role in memory formation, retrieval, and consolidation. Replay-like mechanisms have been incorporated into deep artificial neural networks that learn over time to avoid catastrophic forgetting of previous knowledge. Replay algorithms have been successfully used in a wide range of deep learning methods within supervised, unsupervised, and reinforcement learning paradigms. In this paper, we provide the first comprehensive comparison between replay in the mammalian brain and replay in artificial neural networks. We identify multiple aspects of biological replay that are missing in deep learning systems and hypothesize how they could be utilized to improve artificial neural networks.

Accepted for publication in the MIT Press journal of Neural Computation

References in corpus (9)

• An Overview of Multi-Task Learning in Deep Neural Networks
• Cost-Effective Active Learning for Deep Image Classification
• PathNet: Evolution Channels Gradient Descent in Super Neural Networks
• Efficient Lifelong Learning with A-GEM
• A Comprehensive Study of Class Incremental Learning Algorithms for Visual Tasks
• Measuring Catastrophic Forgetting in Neural Networks
• Neural Episodic Control
• RODEO: Replay for Online Object Detection
• Active Learning for Visual Question Answering: An Empirical Study