Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation

Abstract

Vision-language navigation (VLN) is the task of navigating an embodied agent to carry out natural language instructions inside real 3D environments. In this paper, we study how to address three critical challenges for this task: the cross-modal grounding, the ill-posed feedback, and the generalization problems. First, we propose a novel Reinforced Cross-Modal Matching (RCM) approach that enforces cross-modal grounding both locally and globally via reinforcement learning (RL). Particularly, a matching critic is used to provide an intrinsic reward to encourage global matching between instructions and trajectories, and a reasoning navigator is employed to perform cross-modal grounding in the local visual scene. Evaluation on a VLN benchmark dataset shows that our RCM model significantly outperforms previous methods by 10% on SPL and achieves the new state-of-the-art performance. To improve the generalizability of the learned policy, we further introduce a Self-Supervised Imitation Learning (SIL) method to explore unseen environments by imitating its own past, good decisions. We demonstrate that SIL can approximate a better and more efficient policy, which tremendously minimizes the success rate performance gap between seen and unseen environments (from 30.7% to 11.7%).

arXiv commentCVPR 2019 Oral

References in corpus (19)

• Adam: A Method for Stochastic Optimization
• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• VQA: Visual Question Answering
• VIME: Variational Information Maximizing Exploration
• On Evaluation of Embodied Navigation Agents
• Count-Based Exploration with Neural Density Models
• Matterport3D: Learning from RGB-D Data in Indoor Environments
• #Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learning
• AI2-THOR: An Interactive 3D Environment for Visual AI
• MINOS: Multimodal Indoor Simulator for Navigation in Complex Environments
• Self-Imitation Learning
• Self-Monitoring Navigation Agent via Auxiliary Progress Estimation
• Generation and Comprehension of Unambiguous Object Descriptions
• MAN: Moment Alignment Network for Natural Language Moment Retrieval via Iterative Graph Adjustment
• Turbo Learning for Captionbot and Drawingbot
• Shifting the Baseline: Single Modality Performance on Visual Navigation & QA
• Learning Models for Following Natural Language Directions in Unknown Environments
• The Regretful Agent: Heuristic-Aided Navigation through Progress Estimation
• Tactical Rewind: Self-Correction via Backtracking in Vision-and-Language Navigation

Cited by in corpus (4)

• Trends in Integration of Vision and Language Research: A Survey of Tasks, Datasets, and Methods
• Multi-modal Discriminative Model for Vision-and-Language Navigation
• Enabling Robots to Understand Incomplete Natural Language Instructions Using Commonsense Reasoning
• Walking with MIND: Mental Imagery eNhanceD Embodied QA