Learning and Planning with a Semantic Model

Abstract

Building deep reinforcement learning agents that can generalize and adapt to unseen environments remains a fundamental challenge for AI. This paper describes progresses on this challenge in the context of man-made environments, which are visually diverse but contain intrinsic semantic regularities. We propose a hybrid model-based and model-free approach, LEArning and Planning with Semantics (LEAPS), consisting of a multi-target sub-policy that acts on visual inputs, and a Bayesian model over semantic structures. When placed in an unseen environment, the agent plans with the semantic model to make high-level decisions, proposes the next sub-target for the sub-policy to execute, and updates the semantic model based on new observations. We perform experiments in visual navigation tasks using House3D, a 3D environment that contains diverse human-designed indoor scenes with real-world objects. LEAPS outperforms strong baselines that do not explicitly plan using the semantic content.

arXiv commentsubmitted to ICLR 2019

References in corpus (12)

• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• A Simple Neural Attentive Meta-Learner
• RL${}^2$: Fast Reinforcement Learning via Slow Reinforcement Learning
• Action-Conditional Video Prediction using Deep Networks in Atari Games
• Learning by Playing - Solving Sparse Reward Tasks from Scratch
• Building Generalizable Agents with a Realistic and Rich 3D Environment
• Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World
• Schema Networks: Zero-shot Transfer with a Generative Causal Model of Intuitive Physics
• Self-Supervised Visual Planning with Temporal Skip Connections
• Neural Map: Structured Memory for Deep Reinforcement Learning
• Composable Planning with Attributes
• Memory Augmented Control Networks

Cited by in corpus (5)

• Object Goal Navigation using Goal-Oriented Semantic Exploration
• Shaping embodied agent behavior with activity-context priors from egocentric video
• Continuous Control with Contexts, Provably
• Multimodal Aggregation Approach for Memory Vision-Voice Indoor Navigation with Meta-Learning
• Building Intelligent Autonomous Navigation Agents