Semantic Visual Navigation by Watching YouTube Videos

Abstract

Semantic cues and statistical regularities in real-world environment layouts can improve efficiency for navigation in novel environments. This paper learns and leverages such semantic cues for navigating to objects of interest in novel environments, by simply watching YouTube videos. This is challenging because YouTube videos don't come with labels for actions or goals, and may not even showcase optimal behavior. Our method tackles these challenges through the use of Q-learning on pseudo-labeled transition quadruples (image, action, next image, reward). We show that such off-policy Q-learning from passive data is able to learn meaningful semantic cues for navigation. These cues, when used in a hierarchical navigation policy, lead to improved efficiency at the ObjectGoal task in visually realistic simulations. We observe a relative improvement of 15-83% over end-to-end RL, behavior cloning, and classical methods, while using minimal direct interaction.

arXiv commentNeurIPS 2020. Project website with code, models, and videos: https://matthewchang.github.io/value-learning-from-video

References in corpus (9)

• On Evaluation of Embodied Navigation Agents
• Learning to Navigate in Complex Environments
• Matterport3D: Learning from RGB-D Data in Indoor Environments
• Generative Adversarial Imitation from Observation
• Imitating Latent Policies from Observation
• Visual Memory for Robust Path Following
• Learning Actionable Representations from Visual Observations
• State-only Imitation with Transition Dynamics Mismatch
• Perceptual Values from Observation

Cited by in corpus (3)

• Frontier Semantic Exploration for Visual Target Navigation
• Instabilities of Offline RL with Pre-Trained Neural Representation
• Reinforcement Learning with Videos: Combining Offline Observations with Interaction