SOON: Scenario Oriented Object Navigation with Graph-based Exploration

Abstract

The ability to navigate like a human towards a language-guided target from anywhere in a 3D embodied environment is one of the 'holy grail' goals of intelligent robots. Most visual navigation benchmarks, however, focus on navigating toward a target from a fixed starting point, guided by an elaborate set of instructions that depicts step-by-step. This approach deviates from real-world problems in which human-only describes what the object and its surrounding look like and asks the robot to start navigation from anywhere. Accordingly, in this paper, we introduce a Scenario Oriented Object Navigation (SOON) task. In this task, an agent is required to navigate from an arbitrary position in a 3D embodied environment to localize a target following a scene description. To give a promising direction to solve this task, we propose a novel graph-based exploration (GBE) method, which models the navigation state as a graph and introduces a novel graph-based exploration approach to learn knowledge from the graph and stabilize training by learning sub-optimal trajectories. We also propose a new large-scale benchmark named From Anywhere to Object (FAO) dataset. To avoid target ambiguity, the descriptions in FAO provide rich semantic scene information includes: object attribute, object relationship, region description, and nearby region description. Our experiments reveal that the proposed GBE outperforms various state-of-the-arts on both FAO and R2R datasets. And the ablation studies on FAO validates the quality of the dataset.

References in corpus (15)

• End to End Learning for Self-Driving Cars
• On Evaluation of Embodied Navigation Agents
• Generative Adversarial Imitation Learning
• Matterport3D: Learning from RGB-D Data in Indoor Environments
• Object Goal Navigation using Goal-Oriented Semantic Exploration
• Learning to Explore using Active Neural SLAM
• MINOS: Multimodal Indoor Simulator for Navigation in Complex Environments
• Self-Monitoring Navigation Agent via Auxiliary Progress Estimation
• Vision-and-Dialog Navigation
• Neural SLAM: Learning to Explore with External Memory
• Chasing Ghosts: Instruction Following as Bayesian State Tracking
• General Evaluation for Instruction Conditioned Navigation using Dynamic Time Warping
• Look Before You Leap: Bridging Model-Free and Model-Based Reinforcement Learning for Planned-Ahead Vision-and-Language Navigation
• Evolving Graphical Planner: Contextual Global Planning for Vision-and-Language Navigation
• Learning to Visually Navigate in Photorealistic Environments Without any Supervision