Automatic Recall Machines: Internal Replay, Continual Learning and the Brain
arXiv:2006.12323
Abstract
Replay in neural networks involves training on sequential data with memorized samples, which counteracts forgetting of previous behavior caused by non-stationarity. We present a method where these auxiliary samples are generated on the fly, given only the model that is being trained for the assessed objective, without extraneous buffers or generator networks. Instead the implicit memory of learned samples within the assessed model itself is exploited. Furthermore, whereas existing work focuses on reinforcing the full seen data distribution, we show that optimizing for not forgetting calls for the generation of samples that are specialized to each real training batch, which is more efficient and scalable. We consider high-level parallels with the brain, notably the use of a single model for inference and recall, the dependency of recalled samples on the current environment batch, top-down modulation of activations and learning, abstract recall, and the dependency between the degree to which a task is learned and the degree to which it is recalled. These characteristics emerge naturally from the method without being controlled for.
arXiv commentNeurIPS 2020 Workshop on BabyMind
References in corpus (16)
- Distilling the Knowledge in a Neural Network
- Overcoming catastrophic forgetting in neural networks
- Playing Atari with Deep Reinforcement Learning
- Continual Learning Through Synaptic Intelligence
- Efficient Lifelong Learning with A-GEM
- Three scenarios for continual learning
- On Tiny Episodic Memories in Continual Learning
- FearNet: Brain-Inspired Model for Incremental Learning
- Overcoming Catastrophic Forgetting by Incremental Moment Matching
- Generative replay with feedback connections as a general strategy for continual learning
- Towards Robust Evaluations of Continual Learning
- Deep Generative Dual Memory Network for Continual Learning
- Lifelong Generative Modeling
- Pseudo-Recursal: Solving the Catastrophic Forgetting Problem in Deep Neural Networks
- Meta Continual Learning
- Continual Classification Learning Using Generative Models