DisCoRL: Continual Reinforcement Learning via Policy Distillation

Abstract

In multi-task reinforcement learning there are two main challenges: at training time, the ability to learn different policies with a single model; at test time, inferring which of those policies applying without an external signal. In the case of continual reinforcement learning a third challenge arises: learning tasks sequentially without forgetting the previous ones. In this paper, we tackle these challenges by proposing DisCoRL, an approach combining state representation learning and policy distillation. We experiment on a sequence of three simulated 2D navigation tasks with a 3 wheel omni-directional robot. Moreover, we tested our approach's robustness by transferring the final policy into a real life setting. The policy can solve all tasks and automatically infer which one to run.

arXiv commentarXiv admin note: text overlap with arXiv:1906.04452

References in corpus (10)

• Overcoming catastrophic forgetting in neural networks
• Categorical Reparameterization with Gumbel-Softmax
• Continual Learning Through Synaptic Intelligence
• PathNet: Evolution Channels Gradient Descent in Super Neural Networks
• State Representation Learning for Control: An Overview
• Transfer from Simulation to Real World through Learning Deep Inverse Dynamics Model
• Sim-to-Real Reinforcement Learning for Deformable Object Manipulation
• Life-Long Disentangled Representation Learning with Cross-Domain Latent Homologies
• Memory Replay GANs: learning to generate images from new categories without forgetting
• S-RL Toolbox: Environments, Datasets and Evaluation Metrics for State Representation Learning

Cited by in corpus (10)

• Acquisition of Chess Knowledge in AlphaZero
• Catastrophic Interference in Reinforcement Learning: A Solution Based on Context Division and Knowledge Distillation
• Continuum: Simple Management of Complex Continual Learning Scenarios
• Tackling Catastrophic Forgetting and Background Shift in Continual Semantic Segmentation
• Continual Policy Distillation of Reinforcement Learning-based Controllers for Soft Robotic In-Hand Manipulation
• A Decentralized Policy Gradient Approach to Multi-task Reinforcement Learning
• Understanding Continual Learning Settings with Data Distribution Drift Analysis
• DREAM Architecture: a Developmental Approach to Open-Ended Learning in Robotics
• Sequoia: A Software Framework to Unify Continual Learning Research
• A Simple Approach to Continual Learning by Transferring Skill Parameters