Transfer Learning by Modeling a Distribution over Policies

Abstract

Exploration and adaptation to new tasks in a transfer learning setup is a central challenge in reinforcement learning. In this work, we build on the idea of modeling a distribution over policies in a Bayesian deep reinforcement learning setup to propose a transfer strategy. Recent works have shown to induce diversity in the learned policies by maximizing the entropy of a distribution of policies (Bachman et al., 2018; Garnelo et al., 2018) and thus, we postulate that our proposed approach leads to faster exploration resulting in improved transfer learning. We support our hypothesis by demonstrating favorable experimental results on a variety of settings on fully-observable GridWorld and partially observable MiniGrid (Chevalier-Boisvert et al., 2018) environments.

arXiv commentAccepted at the ICML 2019 workshop on Multi-Task and Lifelong Reinforcement Learning

References in corpus (8)

• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
• VIME: Variational Information Maximizing Exploration
• Bridging the Gap Between Value and Policy Based Reinforcement Learning
• Stochastic Neural Networks for Hierarchical Reinforcement Learning
• A unified view of entropy-regularized Markov decision processes
• Latent Space Policies for Hierarchical Reinforcement Learning
• Safe Option-Critic: Learning Safety in the Option-Critic Architecture
• VFunc: a Deep Generative Model for Functions