Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learning

Abstract

Two less addressed issues of deep reinforcement learning are (1) lack of generalization capability to new target goals, and (2) data inefficiency i.e., the model requires several (and often costly) episodes of trial and error to converge, which makes it impractical to be applied to real-world scenarios. In this paper, we address these two issues and apply our model to the task of target-driven visual navigation. To address the first issue, we propose an actor-critic model whose policy is a function of the goal as well as the current state, which allows to better generalize. To address the second issue, we propose AI2-THOR framework, which provides an environment with high-quality 3D scenes and physics engine. Our framework enables agents to take actions and interact with objects. Hence, we can collect a huge number of training samples efficiently. We show that our proposed method (1) converges faster than the state-of-the-art deep reinforcement learning methods, (2) generalizes across targets and across scenes, (3) generalizes to a real robot scenario with a small amount of fine-tuning (although the model is trained in simulation), (4) is end-to-end trainable and does not need feature engineering, feature matching between frames or 3D reconstruction of the environment. The supplementary video can be accessed at the following link: https://youtu.be/SmBxMDiOrvs.

References in corpus (4)

• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• Virtual Worlds as Proxy for Multi-Object Tracking Analysis
• Policy Distillation
• Learning Physical Intuition of Block Towers by Example

Cited by in corpus (38)

• An Introduction to Deep Reinforcement Learning
• Semi-supervised Deep Reinforcement Learning in Support of IoT and Smart City Services
• Neural SLAM: Learning to Explore with External Memory
• Virtual-to-real Deep Reinforcement Learning: Continuous Control of Mobile Robots for Mapless Navigation
• Spatial Action Maps for Mobile Manipulation
• Intention-Net: Integrating Planning and Deep Learning for Goal-Directed Autonomous Navigation
• Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks
• A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation
• Unifying Map and Landmark Based Representations for Visual Navigation
• Deep Reinforcement Learning-based Image Captioning with Embedding Reward
• Learned Visual Navigation for Under-Canopy Agricultural Robots
• Deep Variation-structured Reinforcement Learning for Visual Relationship and Attribute Detection
• ViKiNG: Vision-Based Kilometer-Scale Navigation with Geographic Hints
• A Survey on Deep Learning Methods for Robot Vision
• EvalAI: Towards Better Evaluation Systems for AI Agents
• Detecting Adversarial Attacks on Neural Network Policies with Visual Foresight
• Deep Reinforcement Learning with Successor Features for Navigation across Similar Environments
• Realtime Collision Avoidance for Mobile Robots in Dense Crowds using Implicit Multi-sensor Fusion and Deep Reinforcement Learning
• Learning to Fly by Crashing
• Collaborative Deep Reinforcement Learning
• The Wireless Control Plane: An Overview and Directions for Future Research
• Omnidirectional CNN for Visual Place Recognition and Navigation
• Cross-Domain Perceptual Reward Functions
• An A* Curriculum Approach to Reinforcement Learning for RGBD Indoor Robot Navigation
• Model-based Behavioral Cloning with Future Image Similarity Learning
• SGoLAM: Simultaneous Goal Localization and Mapping for Multi-Object Goal Navigation
• Probing Emergent Semantics in Predictive Agents via Question Answering
• Understanding Convolutional Networks with APPLE : Automatic Patch Pattern Labeling for Explanation
• Cross-Domain Self-supervised Multi-task Feature Learning using Synthetic Imagery
• SeGAN: Segmenting and Generating the Invisible
• Role Playing Learning for Socially Concomitant Mobile Robot Navigation
• Avoidance of Manual Labeling in Robotic Autonomous Navigation Through Multi-Sensory Semi-Supervised Learning
• Deep Reinforcement Learning for Portfolio Optimization using Latent Feature State Space (LFSS) Module
• DeepMPCVS: Deep Model Predictive Control for Visual Servoing
• DeepNav: Learning to Navigate Large Cities
• Image Captioning based on Deep Reinforcement Learning
• From Virtual to Real World Visual Perception using Domain Adaptation -- The DPM as Example
• Transfer Reinforcement Learning across Homotopy Classes