Retouchdown: Adding Touchdown to StreetLearn as a Shareable Resource for Language Grounding Tasks in Street View

Abstract

The Touchdown dataset (Chen et al., 2019) provides instructions by human annotators for navigation through New York City streets and for resolving spatial descriptions at a given location. To enable the wider research community to work effectively with the Touchdown tasks, we are publicly releasing the 29k raw Street View panoramas needed for Touchdown. We follow the process used for the StreetLearn data release (Mirowski et al., 2019) to check panoramas for personally identifiable information and blur them as necessary. These have been added to the StreetLearn dataset and can be obtained via the same process as used previously for StreetLearn. We also provide a reference implementation for both of the Touchdown tasks: vision and language navigation (VLN) and spatial description resolution (SDR). We compare our model results to those given in Chen et al. (2019) and show that the panoramas we have added to StreetLearn fully support both Touchdown tasks and can be used effectively for further research and comparison.

References in corpus (5)

• HOGWILD!: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent
• Matterport3D: Learning from RGB-D Data in Indoor Environments
• Grounded Language Learning in a Simulated 3D World
• The StreetLearn Environment and Dataset
• VALAN: Vision and Language Agent Navigation

Cited by in corpus (9)

• AllenAct: A Framework for Embodied AI Research
• Sim-to-Real Transfer for Vision-and-Language Navigation
• Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Grounding
• On the Evaluation of Vision-and-Language Navigation Instructions
• Multimodal Text Style Transfer for Outdoor Vision-and-Language Navigation
• Where Are You? Localization from Embodied Dialog
• SILG: The Multi-environment Symbolic Interactive Language Grounding Benchmark
• PanGEA: The Panoramic Graph Environment Annotation Toolkit
• Language-Aligned Waypoint (LAW) Supervision for Vision-and-Language Navigation in Continuous Environments