Explore the Potential Performance of Vision-and-Language Navigation Model: a Snapshot Ensemble Method
arXiv:2111.14267
Abstract
Vision-and-Language Navigation (VLN) is a challenging task in the field of artificial intelligence. Although massive progress has been made in this task over the past few years attributed to breakthroughs in deep vision and language models, it remains tough to build VLN models that can generalize as well as humans. In this paper, we provide a new perspective to improve VLN models. Based on our discovery that snapshots of the same VLN model behave significantly differently even when their success rates are relatively the same, we propose a snapshot-based ensemble solution that leverages predictions among multiple snapshots. Constructed on the snapshots of the existing state-of-the-art (SOTA) model BERT and our past-action-aware modification, our proposed ensemble achieves the new SOTA performance in the R2R dataset challenge in Navigation Error (NE) and Success weighted by Path Length (SPL).
arXiv comment7 pages
References in corpus (14)
- SGDR: Stochastic Gradient Descent with Warm Restarts
- Temporal Ensembling for Semi-Supervised Learning
- AI2-THOR: An Interactive 3D Environment for Visual AI
- Building Generalizable Agents with a Realistic and Rich 3D Environment
- LXMERT: Learning Cross-Modality Encoder Representations from Transformers
- Self-Monitoring Navigation Agent via Auxiliary Progress Estimation
- HoME: a Household Multimodal Environment
- CHALET: Cornell House Agent Learning Environment
- Self-ensembling for visual domain adaptation
- Language and Visual Entity Relationship Graph for Agent Navigation
- The Regretful Agent: Heuristic-Aided Navigation through Progress Estimation
- Are You Looking? Grounding to Multiple Modalities in Vision-and-Language Navigation
- Robust Navigation with Language Pretraining and Stochastic Sampling
- Sub-Instruction Aware Vision-and-Language Navigation