A Critical Investigation of Deep Reinforcement Learning for Navigation

Abstract

The navigation problem is classically approached in two steps: an exploration step, where map-information about the environment is gathered; and an exploitation step, where this information is used to navigate efficiently. Deep reinforcement learning (DRL) algorithms, alternatively, approach the problem of navigation in an end-to-end fashion. Inspired by the classical approach, we ask whether DRL algorithms are able to inherently explore, gather and exploit map-information over the course of navigation. We build upon Mirowski et al. [2017] work and introduce a systematic suite of experiments that vary three parameters: the agent's starting location, the agent's target location, and the maze structure. We choose evaluation metrics that explicitly measure the algorithm's ability to gather and exploit map-information. Our experiments show that when trained and tested on the same maps, the algorithm successfully gathers and exploits map-information. However, when trained and tested on different sets of maps, the algorithm fails to transfer the ability to gather and exploit map-information to unseen maps. Furthermore, we find that when the goal location is randomized and the map is kept static, the algorithm is able to gather and exploit map-information but the exploitation is far from optimal. We open-source our experimental suite in the hopes that it serves as a framework for the comparison of future algorithms and leads to the discovery of robust alternatives to classical navigation methods.

References in corpus (6)

• Playing Atari with Deep Reinforcement Learning
• Learning to Navigate in Complex Environments
• Control of Memory, Active Perception, and Action in Minecraft
• Schema Networks: Zero-shot Transfer with a Generative Causal Model of Intuitive Physics
• Neural Map: Structured Memory for Deep Reinforcement Learning
• Deep Successor Reinforcement Learning

Cited by in corpus (5)

• Deep Reinforcement Learning (DRL): Another Perspective for Unsupervised Wireless Localization
• Learning to Navigate in Cities Without a Map
• Improving Target-driven Visual Navigation with Attention on 3D Spatial Relationships
• Floyd-Warshall Reinforcement Learning: Learning from Past Experiences to Reach New Goals
• The RobotSlang Benchmark: Dialog-guided Robot Localization and Navigation