Improving Target-driven Visual Navigation with Attention on 3D Spatial Relationships

Abstract

Embodied artificial intelligence (AI) tasks shift from tasks focusing on internet images to active settings involving embodied agents that perceive and act within 3D environments. In this paper, we investigate the target-driven visual navigation using deep reinforcement learning (DRL) in 3D indoor scenes, whose navigation task aims to train an agent that can intelligently make a series of decisions to arrive at a pre-specified target location from any possible starting positions only based on egocentric views. However, most navigation methods currently struggle against several challenging problems, such as data efficiency, automatic obstacle avoidance, and generalization. Generalization problem means that agent does not have the ability to transfer navigation skills learned from previous experience to unseen targets and scenes. To address these issues, we incorporate two designs into classic DRL framework: attention on 3D knowledge graph (KG) and target skill extension (TSE) module. On the one hand, our proposed method combines visual features and 3D spatial representations to learn navigation policy. On the other hand, TSE module is used to generate sub-targets which allow agent to learn from failures. Specifically, our 3D spatial relationships are encoded through recently popular graph convolutional network (GCN). Considering the real world settings, our work also considers open action and adds actionable targets into conventional navigation situations. Those more difficult settings are applied to test whether DRL agent really understand its task, navigating environment, and can carry out reasoning. Our experiments, performed in the AI2-THOR, show that our model outperforms the baselines in both SR and SPL metrics, and improves generalization ability across targets and scenes.

12 pages, 9 figures

References in corpus (17)

• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• YOLOv3: An Incremental Improvement
• Dueling Network Architectures for Deep Reinforcement Learning
• On Evaluation of Embodied Navigation Agents
• Learning to Navigate in Complex Environments
• Matterport3D: Learning from RGB-D Data in Indoor Environments
• Hindsight Experience Replay
• AI2-THOR: An Interactive 3D Environment for Visual AI
• Reinforcement Learning with Unsupervised Auxiliary Tasks
• Building Generalizable Agents with a Realistic and Rich 3D Environment
• Control of Memory, Active Perception, and Action in Minecraft
• Learning Conditioned Graph Structures for Interpretable Visual Question Answering
• Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learning
• Listen, Attend, and Walk: Neural Mapping of Navigational Instructions to Action Sequences
• Gated-Attention Architectures for Task-Oriented Language Grounding
• Deep Successor Reinforcement Learning
• A Critical Investigation of Deep Reinforcement Learning for Navigation

Cited by in corpus (6)

• From Seeing to Moving: A Survey on Learning for Visual Indoor Navigation (VIN)
• Deep Learning for Embodied Vision Navigation: A Survey
• A review of mobile robot motion planning methods: from classical motion planning workflows to reinforcement learning-based architectures
• Semantic Environment Atlas for Object-Goal Navigation
• Towards Target-Driven Visual Navigation in Indoor Scenes via Generative Imitation Learning
• VSGM -- Enhance robot task understanding ability through visual semantic graph