Deep Sets for Generalization in RL
arXiv:2003.09443
Abstract
This paper investigates the idea of encoding object-centered representations in the design of the reward function and policy architectures of a language-guided reinforcement learning agent. This is done using a combination of object-wise permutation invariant networks inspired from Deep Sets and gated-attention mechanisms. In a 2D procedurally-generated world where agents targeting goals in natural language navigate and interact with objects, we show that these architectures demonstrate strong generalization capacities to out-of-distribution goals. We study the generalization to varying numbers of objects at test time and further extend the object-centered architectures to goals involving relational reasoning.
15 pages, 10 figures, published as a workshop Paper at ICLR: Beyond tabula rasa in RL (BeTR-RL). arXiv admin note: substantial text overlap with arXiv:2002.09253
References in corpus (8)
- Multi-Object Representation Learning with Iterative Variational Inference
- MONet: Unsupervised Scene Decomposition and Representation
- Relational Deep Reinforcement Learning
- From Language to Goals: Inverse Reinforcement Learning for Vision-Based Instruction Following
- Learning to Understand Goal Specifications by Modelling Reward
- Language as a Cognitive Tool to Imagine Goals in Curiosity-Driven Exploration
- A Hitchhiker's Guide to Statistical Comparisons of Reinforcement Learning Algorithms
- Unicorn: Continual Learning with a Universal, Off-policy Agent