Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directions
arXiv:2203.12667 · doi:10.18653/v1/2022.acl-long.524
Abstract
A long-term goal of AI research is to build intelligent agents that can communicate with humans in natural language, perceive the environment, and perform real-world tasks. Vision-and-Language Navigation (VLN) is a fundamental and interdisciplinary research topic towards this goal, and receives increasing attention from natural language processing, computer vision, robotics, and machine learning communities. In this paper, we review contemporary studies in the emerging field of VLN, covering tasks, evaluation metrics, methods, etc. Through structured analysis of current progress and challenges, we highlight the limitations of current VLN and opportunities for future work. This paper serves as a thorough reference for the VLN research community.
arXiv comment19 pages. Accepted to ACL 2022
References in corpus (26)
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks
- On Evaluation of Embodied Navigation Agents
- The Replica Dataset: A Digital Replica of Indoor Spaces
- Building Generalizable Agents with a Realistic and Rich 3D Environment
- Speaker-Follower Models for Vision-and-Language Navigation
- Self-Monitoring Navigation Agent via Auxiliary Progress Estimation
- Vision-and-Dialog Navigation
- Language-guided Navigation via Cross-Modal Grounding and Alternate Adversarial Learning
- From Language to Goals: Inverse Reinforcement Learning for Vision-Based Instruction Following
- Gibson Env: Real-World Perception for Embodied Agents
- Talk2Nav: Long-Range Vision-and-Language Navigation with Dual Attention and Spatial Memory
- Chasing Ghosts: Instruction Following as Bayesian State Tracking
- DialFRED: Dialogue-Enabled Agents for Embodied Instruction Following
- The StreetLearn Environment and Dataset
- Learning to Map Natural Language Instructions to Physical Quadcopter Control using Simulated Flight
- MeetUp! A Corpus of Joint Activity Dialogues in a Visual Environment
- Embodied Vision-and-Language Navigation with Dynamic Convolutional Filters
- RMM: A Recursive Mental Model for Dialog Navigation
- Deep Learning for Embodied Vision Navigation: A Survey
- The Road to Know-Where: An Object-and-Room Informed Sequential BERT for Indoor Vision-Language Navigation
- Multi-View Learning for Vision-and-Language Navigation
- Diagnosing Vision-and-Language Navigation: What Really Matters
- Self-supervised 3D Semantic Representation Learning for Vision-and-Language Navigation
- A Framework for Learning to Request Rich and Contextually Useful Information from Humans
- Rethinking the Spatial Route Prior in Vision-and-Language Navigation
Cited by in corpus (11)
- VLP: A Survey on Vision-Language Pre-training
- Language-Grounded Dynamic Scene Graphs for Interactive Object Search with Mobile Manipulation
- DialFRED: Dialogue-Enabled Agents for Embodied Instruction Following
- Open Scene Graphs for Open-World Object-Goal Navigation
- Language to Map: Topological map generation from natural language path instructions
- Hierarchical Path-planning from Speech Instructions with Spatial Concept-based Topometric Semantic Mapping
- Multimodal Speech Recognition for Language-Guided Embodied Agents
- Memory-Maze: Scenario Driven Visual Language Navigation Benchmark for Guiding Blind People
- Real-world Instance-specific Image Goal Navigation: Bridging Domain Gaps via Contrastive Learning
- Revolutionizing Turn-by-Turn Navigation with Cloud-Edge Deep Learning
- CMAR-search: Commonsense and Memory Augmented Reasoning for Object Search in Dynamic Interactive Environments