An Optimal Online Method of Selecting Source Policies for Reinforcement Learning

Abstract

Transfer learning significantly accelerates the reinforcement learning process by exploiting relevant knowledge from previous experiences. The problem of optimally selecting source policies during the learning process is of great importance yet challenging. There has been little theoretical analysis of this problem. In this paper, we develop an optimal online method to select source policies for reinforcement learning. This method formulates online source policy selection as a multi-armed bandit problem and augments Q-learning with policy reuse. We provide theoretical guarantees of the optimal selection process and convergence to the optimal policy. In addition, we conduct experiments on a grid-based robot navigation domain to demonstrate its efficiency and robustness by comparing to the state-of-the-art transfer learning method.

Cited by in corpus (6)

• AC-Teach: A Bayesian Actor-Critic Method for Policy Learning with an Ensemble of Suboptimal Teachers
• Efficient Deep Reinforcement Learning via Adaptive Policy Transfer
• LISPR: An Options Framework for Policy Reuse with Reinforcement Learning
• TempLe: Learning Template of Transitions for Sample Efficient Multi-task RL
• Discovering Generalizable Skills via Automated Generation of Diverse Tasks
• Lifetime policy reuse and the importance of task capacity