Continual Reinforcement Learning with Complex Synapses

Abstract

Unlike humans, who are capable of continual learning over their lifetimes, artificial neural networks have long been known to suffer from a phenomenon known as catastrophic forgetting, whereby new learning can lead to abrupt erasure of previously acquired knowledge. Whereas in a neural network the parameters are typically modelled as scalar values, an individual synapse in the brain comprises a complex network of interacting biochemical components that evolve at different timescales. In this paper, we show that by equipping tabular and deep reinforcement learning agents with a synaptic model that incorporates this biological complexity (Benna & Fusi, 2016), catastrophic forgetting can be mitigated at multiple timescales. In particular, we find that as well as enabling continual learning across sequential training of two simple tasks, it can also be used to overcome within-task forgetting by reducing the need for an experience replay database.

arXiv commentAccepted at ICML 2018

References in corpus (7)

• Adam: A Method for Stochastic Optimization
• Continuous control with deep reinforcement learning
• Overcoming catastrophic forgetting in neural networks
• Continual Learning Through Synaptic Intelligence
• An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks
• PathNet: Evolution Channels Gradient Descent in Super Neural Networks
• Continual Learning with Deep Generative Replay

Cited by in corpus (6)

• Functional Regularisation for Continual Learning with Gaussian Processes
• Orthogonal Gradient Descent for Continual Learning
• Deep Reinforcement and InfoMax Learning
• Continual Reinforcement Learning with Diversity Exploration and Adversarial Self-Correction
• Merging Deterministic Policy Gradient Estimations with Varied Bias-Variance Tradeoff for Effective Deep Reinforcement Learning
• GRIm-RePR: Prioritising Generating Important Features for Pseudo-Rehearsal