Synthesized Policies for Transfer and Adaptation across Tasks and Environments

Abstract

The ability to transfer in reinforcement learning is key towards building an agent of general artificial intelligence. In this paper, we consider the problem of learning to simultaneously transfer across both environments (ENV) and tasks (TASK), probably more importantly, by learning from only sparse (ENV, TASK) pairs out of all the possible combinations. We propose a novel compositional neural network architecture which depicts a meta rule for composing policies from the environment and task embeddings. Notably, one of the main challenges is to learn the embeddings jointly with the meta rule. We further propose new training methods to disentangle the embeddings, making them both distinctive signatures of the environments and tasks and effective building blocks for composing the policies. Experiments on GridWorld and Thor, of which the agent takes as input an egocentric view, show that our approach gives rise to high success rates on all the (ENV, TASK) pairs after learning from only 40% of them.

arXiv commentpresented at NeurIPS 2018 as a Spotlight

References in corpus (5)

• Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm
• StarCraft II: A New Challenge for Reinforcement Learning
• AI2-THOR: An Interactive 3D Environment for Visual AI
• Modular Multitask Reinforcement Learning with Policy Sketches
• Deep Successor Reinforcement Learning

Cited by in corpus (5)

• Learning Generalizable Visual Representations via Interactive Gameplay
• BabyWalk: Going Farther in Vision-and-Language Navigation by Taking Baby Steps
• Skill Transfer in Deep Reinforcement Learning under Morphological Heterogeneity
• Generalized Hidden Parameter MDPs Transferable Model-based RL in a Handful of Trials
• Open Compound Domain Adaptation