Bayesian Relational Memory for Semantic Visual Navigation

Abstract

We introduce a new memory architecture, Bayesian Relational Memory (BRM), to improve the generalization ability for semantic visual navigation agents in unseen environments, where an agent is given a semantic target to navigate towards. BRM takes the form of a probabilistic relation graph over semantic entities (e.g., room types), which allows (1) capturing the layout prior from training environments, i.e., prior knowledge, (2) estimating posterior layout at test time, i.e., memory update, and (3) efficient planning for navigation, altogether. We develop a BRM agent consisting of a BRM module for producing sub-goals and a goal-conditioned locomotion module for control. When testing in unseen environments, the BRM agent outperforms baselines that do not explicitly utilize the probabilistic relational memory structure

arXiv commentAccepted at ICCV 2019

References in corpus (20)

• A Simple Neural Attentive Meta-Learner
• Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation
• RL${}^2$: Fast Reinforcement Learning via Slow Reinforcement Learning
• On Evaluation of Embodied Navigation Agents
• Learning to Navigate in Complex Environments
• AI2-THOR: An Interactive 3D Environment for Visual AI
• Reinforcement Learning with Unsupervised Auxiliary Tasks
• Learning by Playing - Solving Sparse Reward Tasks from Scratch
• Building Generalizable Agents with a Realistic and Rich 3D Environment
• Episodic Curiosity through Reachability
• MINOS: Multimodal Indoor Simulator for Navigation in Complex Environments
• Gated-Attention Architectures for Task-Oriented Language Grounding
• Learning Plannable Representations with Causal InfoGAN
• Unifying Map and Landmark Based Representations for Visual Navigation
• segDeepM: Exploiting Segmentation and Context in Deep Neural Networks for Object Detection
• Benchmarking Classic and Learned Navigation in Complex 3D Environments
• Composable Planning with Attributes
• Probabilistic Neural-symbolic Models for Interpretable Visual Question Answering
• Blindfold Baselines for Embodied QA
• A Behavioral Approach to Visual Navigation with Graph Localization Networks