Deep Reinforcement Learning with Successor Features for Navigation across Similar Environments

Abstract

In this paper we consider the problem of robot navigation in simple maze-like environments where the robot has to rely on its onboard sensors to perform the navigation task. In particular, we are interested in solutions to this problem that do not require localization, mapping or planning. Additionally, we require that our solution can quickly adapt to new situations (e.g., changing navigation goals and environments). To meet these criteria we frame this problem as a sequence of related reinforcement learning tasks. We propose a successor feature based deep reinforcement learning algorithm that can learn to transfer knowledge from previously mastered navigation tasks to new problem instances. Our algorithm substantially decreases the required learning time after the first task instance has been solved, which makes it easily adaptable to changing environments. We validate our method in both simulated and real robot experiments with a Robotino and compare it to a set of baseline methods including classical planning-based navigation.

Camera ready version for IROS 2017

References in corpus (4)

• Distilling the Knowledge in a Neural Network
• Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learning
• Towards Cognitive Exploration through Deep Reinforcement Learning for Mobile Robots
• A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation

Cited by in corpus (11)

• Neural SLAM: Learning to Explore with External Memory
• Virtual-to-real Deep Reinforcement Learning: Continuous Control of Mobile Robots for Mapless Navigation
• One-Shot Reinforcement Learning for Robot Navigation with Interactive Replay
• Advantages and Limitations of using Successor Features for Transfer in Reinforcement Learning
• Towards Optimally Decentralized Multi-Robot Collision Avoidance via Deep Reinforcement Learning
• Universal Successor Features Approximators
• The AdobeIndoorNav Dataset: Towards Deep Reinforcement Learning based Real-world Indoor Robot Visual Navigation
• On Reward Shaping for Mobile Robot Navigation: A Reinforcement Learning and SLAM Based Approach
• DiGrad: Multi-Task Reinforcement Learning with Shared Actions
• Transfer with Model Features in Reinforcement Learning
• Learning with Training Wheels: Speeding up Training with a Simple Controller for Deep Reinforcement Learning