Progressive Reinforcement Learning with Distillation for Multi-Skilled Motion Control

Abstract

Deep reinforcement learning has demonstrated increasing capabilities for continuous control problems, including agents that can move with skill and agility through their environment. An open problem in this setting is that of developing good strategies for integrating or merging policies for multiple skills, where each individual skill is a specialist in a specific skill and its associated state distribution. We extend policy distillation methods to the continuous action setting and leverage this technique to combine expert policies, as evaluated in the domain of simulated bipedal locomotion across different classes of terrain. We also introduce an input injection method for augmenting an existing policy network to exploit new input features. Lastly, our method uses transfer learning to assist in the efficient acquisition of new skills. The combination of these methods allows a policy to be incrementally augmented with new skills. We compare our progressive learning and integration via distillation (PLAID) method against three alternative baselines.

arXiv comment15 pages, Conference paper

References in corpus (10)

• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation
• Continual Learning with Deep Generative Replay
• Actor-Mimic: Deep Multitask and Transfer Reinforcement Learning
• Distral: Robust Multitask Reinforcement Learning
• Net2Net: Accelerating Learning via Knowledge Transfer
• Learning human behaviors from motion capture by adversarial imitation
• Learning and Transfer of Modulated Locomotor Controllers
• POWERPLAY: Training an Increasingly General Problem Solver by Continually Searching for the Simplest Still Unsolvable Problem
• First Experiments with PowerPlay

Cited by in corpus (7)

• Learning to Walk in the Real World with Minimal Human Effort
• Robust Recovery Controller for a Quadrupedal Robot using Deep Reinforcement Learning
• Meta Continual Learning
• Continual Model-Based Reinforcement Learning with Hypernetworks
• One Big Net For Everything
• Transfer Heterogeneous Knowledge Among Peer-to-Peer Teammates: A Model Distillation Approach
• Scaling shared model governance via model splitting