Adaptive Policy Transfer in Reinforcement Learning

Abstract

Efficient and robust policy transfer remains a key challenge for reinforcement learning to become viable for real-wold robotics. Policy transfer through warm initialization, imitation, or interacting over a large set of agents with randomized instances, have been commonly applied to solve a variety of Reinforcement Learning tasks. However, this seems far from how skill transfer happens in the biological world: Humans and animals are able to quickly adapt the learned behaviors between similar tasks and learn new skills when presented with new situations. Here we seek to answer the question: Will learning to combine adaptation and exploration lead to a more efficient transfer of policies between domains? We introduce a principled mechanism that can "Adapt-to-Learn", that is adapt the source policy to learn to solve a target task with significant transition differences and uncertainties. We show that the presented method learns to seamlessly combine learning from adaptation and exploration and leads to a robust policy transfer algorithm with significantly reduced sample complexity in transferring skills between related tasks.

References in corpus (6)

• Trust Region Policy Optimization
• Meta-SGD: Learning to Learn Quickly for Few-Shot Learning
• Emergence of Locomotion Behaviours in Rich Environments
• Transfer from Simulation to Real World through Learning Deep Inverse Dynamics Model
• Sim-to-Real Transfer of Accurate Grasping with Eye-In-Hand Observations and Continuous Control
• Mutual Alignment Transfer Learning