Semi-parametric Topological Memory for Navigation

Abstract

We introduce a new memory architecture for navigation in previously unseen environments, inspired by landmark-based navigation in animals. The proposed semi-parametric topological memory (SPTM) consists of a (non-parametric) graph with nodes corresponding to locations in the environment and a (parametric) deep network capable of retrieving nodes from the graph based on observations. The graph stores no metric information, only connectivity of locations corresponding to the nodes. We use SPTM as a planning module in a navigation system. Given only 5 minutes of footage of a previously unseen maze, an SPTM-based navigation agent can build a topological map of the environment and use it to confidently navigate towards goals. The average success rate of the SPTM agent in goal-directed navigation across test environments is higher than the best-performing baseline by a factor of three. A video of the agent is available at https://youtu.be/vRF7f4lhswo

arXiv commentPublished at International Conference on Learning Representations (ICLR) 2018. Project website at https://sites.google.com/view/SPTM

References in corpus (5)

• TensorFlow: A system for large-scale machine learning
• Model-Free Episodic Control
• Control of Memory, Active Perception, and Action in Minecraft
• MINOS: Multimodal Indoor Simulator for Navigation in Complex Environments
• Playing Doom with SLAM-Augmented Deep Reinforcement Learning

Cited by in corpus (29)

• On Evaluation of Embodied Navigation Agents
• Episodic Curiosity through Reachability
• Learning Plannable Representations with Causal InfoGAN
• ViKiNG: Vision-Based Kilometer-Scale Navigation with Geographic Hints
• Embodied Visual Navigation with Automatic Curriculum Learning in Real Environments
• Mapping State Space using Landmarks for Universal Goal Reaching
• Rewriting History with Inverse RL: Hindsight Inference for Policy Improvement
• Learning to Reach Goals via Iterated Supervised Learning
• World Model as a Graph: Learning Latent Landmarks for Planning
• A Behavioral Approach to Visual Navigation with Graph Localization Networks
• Learning Object Relation Graph and Tentative Policy for Visual Navigation
• Learning models for visual 3D localization with implicit mapping
• Evolving Graphical Planner: Contextual Global Planning for Vision-and-Language Navigation
• Divide-and-Conquer Monte Carlo Tree Search For Goal-Directed Planning
• Where are the Keys? -- Learning Object-Centric Navigation Policies on Semantic Maps with Graph Convolutional Networks
• Graph Attention Memory for Visual Navigation
• Plan2Vec: Unsupervised Representation Learning by Latent Plans
• Weakly-Supervised Reinforcement Learning for Controllable Behavior
• Learning Object-conditioned Exploration using Distributed Soft Actor Critic
• Temporal Difference Learning with Neural Networks - Study of the Leakage Propagation Problem
• Reinforcement Learning-based Visual Navigation with Information-Theoretic Regularization
• Hierarchical Robot Navigation in Novel Environments using Rough 2-D Maps
• Topological Navigation Graph Framework
• Deep Visual MPC-Policy Learning for Navigation
• Multimodal Aggregation Approach for Memory Vision-Voice Indoor Navigation with Meta-Learning
• NeoNav: Improving the Generalization of Visual Navigation via Generating Next Expected Observations
• Learning View and Target Invariant Visual Servoing for Navigation
• From proprioception to long-horizon planning in novel environments: A hierarchical RL model
• Hierarchical and Partially Observable Goal-driven Policy Learning with Goals Relational Graph