Policy Consolidation for Continual Reinforcement Learning
arXiv:1902.00255
Abstract
We propose a method for tackling catastrophic forgetting in deep reinforcement learning that is \textit{agnostic} to the timescale of changes in the distribution of experiences, does not require knowledge of task boundaries, and can adapt in \textit{continuously} changing environments. In our \textit{policy consolidation} model, the policy network interacts with a cascade of hidden networks that simultaneously remember the agent's policy at a range of timescales and regularise the current policy by its own history, thereby improving its ability to learn without forgetting. We find that the model improves continual learning relative to baselines on a number of continuous control tasks in single-task, alternating two-task, and multi-agent competitive self-play settings.
Accepted at ICML 2019
Cited by in corpus (6)
- Experience Replay for Continual Learning
- A Survey of Reinforcement Learning Algorithms for Dynamically Varying Environments
- Continual World: A Robotic Benchmark For Continual Reinforcement Learning
- Structure in Deep Reinforcement Learning: A Survey and Open Problems
- Continual Reinforcement Learning with Multi-Timescale Replay
- OER: Offline Experience Replay for Continual Offline Reinforcement Learning