Robust Navigation with Language Pretraining and Stochastic Sampling

Abstract

Core to the vision-and-language navigation (VLN) challenge is building robust instruction representations and action decoding schemes, which can generalize well to previously unseen instructions and environments. In this paper, we report two simple but highly effective methods to address these challenges and lead to a new state-of-the-art performance. First, we adapt large-scale pretrained language models to learn text representations that generalize better to previously unseen instructions. Second, we propose a stochastic sampling scheme to reduce the considerable gap between the expert actions in training and sampled actions in test, so that the agent can learn to correct its own mistakes during long sequential action decoding. Combining the two techniques, we achieve a new state of the art on the Room-to-Room benchmark with 6% absolute gain over the previous best result (47% -> 53%) on the Success Rate weighted by Path Length metric.

arXiv comment8 pages, 4 figures, EMNLP 2019

References in corpus (8)

• Sequence to Sequence Learning with Neural Networks
• How transferable are features in deep neural networks?
• Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks
• Matterport3D: Learning from RGB-D Data in Indoor Environments
• Speaker-Follower Models for Vision-and-Language Navigation
• Self-Monitoring Navigation Agent via Auxiliary Progress Estimation
• Learning Models for Following Natural Language Directions in Unknown Environments
• Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environments

Cited by in corpus (4)

• Vision-Language Navigation with Self-Supervised Auxiliary Reasoning Tasks
• Evolving Graphical Planner: Contextual Global Planning for Vision-and-Language Navigation
• Explore the Potential Performance of Vision-and-Language Navigation Model: a Snapshot Ensemble Method
• Curriculum Learning for Vision-and-Language Navigation