Learning to Navigate in Complex Environments

Abstract

Learning to navigate in complex environments with dynamic elements is an important milestone in developing AI agents. In this work we formulate the navigation question as a reinforcement learning problem and show that data efficiency and task performance can be dramatically improved by relying on additional auxiliary tasks leveraging multimodal sensory inputs. In particular we consider jointly learning the goal-driven reinforcement learning problem with auxiliary depth prediction and loop closure classification tasks. This approach can learn to navigate from raw sensory input in complicated 3D mazes, approaching human-level performance even under conditions where the goal location changes frequently. We provide detailed analysis of the agent behaviour, its ability to localise, and its network activity dynamics, showing that the agent implicitly learns key navigation abilities.

11 pages, 5 appendix pages, 11 figures, 3 tables, under review as a conference paper at ICLR 2017

Cited by in corpus (124)

• A Brief Survey of Deep Reinforcement Learning
• Deep Reinforcement Learning: An Overview
• On Evaluation of Embodied Navigation Agents
• Active learning machine learns to create new quantum experiments
• WILDS: A Benchmark of in-the-Wild Distribution Shifts
• Lifelong Federated Reinforcement Learning: A Learning Architecture for Navigation in Cloud Robotic Systems
• Object Goal Navigation using Goal-Oriented Semantic Exploration
• CURL: Contrastive Unsupervised Representations for Reinforcement Learning
• Deep Reinforcement Learning (DRL): Another Perspective for Unsupervised Wireless Localization
• Towards Monocular Vision based Obstacle Avoidance through Deep Reinforcement Learning
• Self-Monitoring Navigation Agent via Auxiliary Progress Estimation
• Neural SLAM: Learning to Explore with External Memory
• A Survey on Deep Learning for Localization and Mapping: Towards the Age of Spatial Machine Intelligence
• Feature Control as Intrinsic Motivation for Hierarchical Reinforcement Learning
• Adapting Auxiliary Losses Using Gradient Similarity
• Curiosity-driven Exploration for Mapless Navigation with Deep Reinforcement Learning
• Language-guided Navigation via Cross-Modal Grounding and Alternate Adversarial Learning
• DeepMDP: Learning Continuous Latent Space Models for Representation Learning
• Unifying Map and Landmark Based Representations for Visual Navigation
• Contingency-Aware Exploration in Reinforcement Learning
• Unsupervised Video Object Segmentation for Deep Reinforcement Learning
• Talk2Nav: Long-Range Vision-and-Language Navigation with Dual Attention and Spatial Memory
• Chasing Ghosts: Instruction Following as Bayesian State Tracking
• Learning to Navigate in Cities Without a Map
• Learning End-to-end Multimodal Sensor Policies for Autonomous Navigation
• The StreetLearn Environment and Dataset
• Limits of End-to-End Learning
• iGibson 1.0: a Simulation Environment for Interactive Tasks in Large Realistic Scenes
• A Deep Policy Inference Q-Network for Multi-Agent Systems
• Bootstrap Latent-Predictive Representations for Multitask Reinforcement Learning
• Embodied Visual Navigation with Automatic Curriculum Learning in Real Environments
• Benchmarking Classic and Learned Navigation in Complex 3D Environments
• Search on the Replay Buffer: Bridging Planning and Reinforcement Learning
• Learning Exploration Policies for Navigation
• VTNet: Visual Transformer Network for Object Goal Navigation
• Active Neural Localization
• Combining Optimal Control and Learning for Visual Navigation in Novel Environments
• Visual pathways from the perspective of cost functions and multi-task deep neural networks
• Learning to Set Waypoints for Audio-Visual Navigation
• Learning Vision-Guided Quadrupedal Locomotion End-to-End with Cross-Modal Transformers
• Observational Learning by Reinforcement Learning
• Benchmarking projective simulation in navigation problems
• Learning Robotic Manipulation of Granular Media
• Multi-task Deep Reinforcement Learning with PopArt
• Semantic Visual Navigation by Watching YouTube Videos
• Discriminative Particle Filter Reinforcement Learning for Complex Partial Observations
• Deep Learning for Scene Classification: A Survey
• A Critical Investigation of Deep Reinforcement Learning for Navigation
• Guided Feature Transformation (GFT): A Neural Language Grounding Module for Embodied Agents
• Improving Target-driven Visual Navigation with Attention on 3D Spatial Relationships
• Return-Based Contrastive Representation Learning for Reinforcement Learning
• Scene Memory Transformer for Embodied Agents in Long-Horizon Tasks
• Lipschitzness Is All You Need To Tame Off-policy Generative Adversarial Imitation Learning
• Vision-Language Navigation with Self-Supervised Auxiliary Reasoning Tasks
• Accelerating Reinforcement Learning through GPU Atari Emulation
• From Seeing to Moving: A Survey on Learning for Visual Indoor Navigation (VIN)
• Auxiliary Tasks and Exploration Enable ObjectNav
• Learning to Navigate in Indoor Environments: from Memorizing to Reasoning
• The Regretful Agent: Heuristic-Aided Navigation through Progress Estimation
• How You Act Tells a Lot: Privacy-Leakage Attack on Deep Reinforcement Learning
• Learning Navigation Subroutines from Egocentric Videos
• Are You Looking? Grounding to Multiple Modalities in Vision-and-Language Navigation
• What Should I Do Now? Marrying Reinforcement Learning and Symbolic Planning
• Learning Actionable Representations from Visual Observations
• Discovering Diverse Multi-Agent Strategic Behavior via Reward Randomization
• Deep Learning for Embodied Vision Navigation: A Survey
• Exploiting temporal consistency for real-time video depth estimation
• Topological Planning with Transformers for Vision-and-Language Navigation
• A Boolean Task Algebra for Reinforcement Learning
• Tactical Rewind: Self-Correction via Backtracking in Vision-and-Language Navigation
• Look, Listen, and Act: Towards Audio-Visual Embodied Navigation
• The ThreeDWorld Transport Challenge: A Visually Guided Task-and-Motion Planning Benchmark for Physically Realistic Embodied AI
• Motion Perception in Reinforcement Learning with Dynamic Objects
• Learning to Visually Navigate in Photorealistic Environments Without any Supervision
• Floyd-Warshall Reinforcement Learning: Learning from Past Experiences to Reach New Goals
• A review of mobile robot motion planning methods: from classical motion planning workflows to reinforcement learning-based architectures
• HALMA: Humanlike Abstraction Learning Meets Affordance in Rapid Problem Solving
• Graph Attention Memory for Visual Navigation
• Learning to Move with Affordance Maps
• Where are the Keys? -- Learning Object-Centric Navigation Policies on Semantic Maps with Graph Convolutional Networks
• SoundSpaces: Audio-Visual Navigation in 3D Environments
• DynaNet: Neural Kalman Dynamical Model for Motion Estimation and Prediction
• Bayesian Relational Memory for Semantic Visual Navigation
• Stratospheric Aerosol Injection as a Deep Reinforcement Learning Problem
• Learning to Navigate in a VUCA Environment: Hierarchical Multi-expert Approach
• Towards continuous control of flippers for a multi-terrain robot using deep reinforcement learning
• Deep execution monitor for robot assistive tasks
• Auxiliary Tasks Speed Up Learning PointGoal Navigation
• Active 6D Multi-Object Pose Estimation in Cluttered Scenarios with Deep Reinforcement Learning
• StarCraft Micromanagement with Reinforcement Learning and Curriculum Transfer Learning
• Active Visual Information Gathering for Vision-Language Navigation
• Scaling Local Control to Large-Scale Topological Navigation
• SGoLAM: Simultaneous Goal Localization and Mapping for Multi-Object Goal Navigation
• Learning Object-conditioned Exploration using Distributed Soft Actor Critic
• Environment Predictive Coding for Embodied Agents
• Unsupervised Emergence of Egocentric Spatial Structure from Sensorimotor Prediction
• Off-Policy Actor-Critic with Shared Experience Replay
• Scene-Intuitive Agent for Remote Embodied Visual Grounding
• Meta-Learning to Cluster
• Bridging the Imitation Gap by Adaptive Insubordination
• Cross-View Policy Learning for Street Navigation
• LeRoP: A Learning-Based Modular Robot Photography Framework
• Vision-Dialog Navigation by Exploring Cross-modal Memory
• Low-pass Recurrent Neural Networks - A memory architecture for longer-term correlation discovery
• A Simple Approach to Continual Learning by Transferring Skill Parameters
• A Visuospatial Dataset for Naturalistic Verb Learning
• Autonomous Curiosity for Real-Time Training Onboard Robotic Agents
• Learning Efficient Multi-Agent Cooperative Visual Exploration
• Are you doing what I say? On modalities alignment in ALFRED
• Learning Autonomous Exploration and Mapping with Semantic Vision
• Learning to Look Around: Intelligently Exploring Unseen Environments for Unknown Tasks
• Semantic Audio-Visual Navigation
• Explanation of Reinforcement Learning Model in Dynamic Multi-Agent System
• Learning Synthetic to Real Transfer for Localization and Navigational Tasks
• Implementing Inductive bias for different navigation tasks through diverse RNN attractors
• Visual Reaction: Learning to Play Catch with Your Drone
• Vision-Guided Quadrupedal Locomotion in the Wild with Multi-Modal Delay Randomization
• Partially Observable Planning and Learning for Systems with Non-Uniform Dynamics
• Scrutinizing and De-Biasing Intuitive Physics with Neural Stethoscopes
• Building Intelligent Autonomous Navigation Agents
• Vision-based deep execution monitoring
• MAPEL: Multi-Agent Pursuer-Evader Learning using Situation Report
• Graph Convolutional Memory using Topological Priors
• Augmenting Automated Game Testing with Deep Reinforcement Learning