Virtual-to-real Deep Reinforcement Learning: Continuous Control of Mobile Robots for Mapless Navigation

Abstract

We present a learning-based mapless motion planner by taking the sparse 10-dimensional range findings and the target position with respect to the mobile robot coordinate frame as input and the continuous steering commands as output. Traditional motion planners for mobile ground robots with a laser range sensor mostly depend on the obstacle map of the navigation environment where both the highly precise laser sensor and the obstacle map building work of the environment are indispensable. We show that, through an asynchronous deep reinforcement learning method, a mapless motion planner can be trained end-to-end without any manually designed features and prior demonstrations. The trained planner can be directly applied in unseen virtual and real environments. The experiments show that the proposed mapless motion planner can navigate the nonholonomic mobile robot to the desired targets without colliding with any obstacles.

video: https://www.youtube.com/watch?v=9AOIwBYIBbs, 6 pages, 9 figures, to appear in he 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2017), final submission version

References in corpus (5)

• Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learning
• Towards Cognitive Exploration through Deep Reinforcement Learning for Mobile Robots
• A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation
• Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updates
• Deep Reinforcement Learning with Successor Features for Navigation across Similar Environments

Cited by in corpus (16)

• A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation
• One-Shot Reinforcement Learning for Robot Navigation with Interactive Replay
• CrowdMove: Autonomous Mapless Navigation in Crowded Scenarios
• Towards Optimally Decentralized Multi-Robot Collision Avoidance via Deep Reinforcement Learning
• Learning Deployable Navigation Policies at Kilometer Scale from a Single Traversal
• The AdobeIndoorNav Dataset: Towards Deep Reinforcement Learning based Real-world Indoor Robot Visual Navigation
• Crowd-Robot Interaction: Crowd-aware Robot Navigation with Attention-based Deep Reinforcement Learning
• Occupancy Map Prediction Using Generative and Fully Convolutional Networks for Vehicle Navigation
• Parameter Sharing Reinforcement Learning Architecture for Multi Agent Driving Behaviors
• Deep Reinforcement Learning to Acquire Navigation Skills for Wheel-Legged Robots in Complex Environments
• Towards continuous control of flippers for a multi-terrain robot using deep reinforcement learning
• Avoidance of Manual Labeling in Robotic Autonomous Navigation Through Multi-Sensory Semi-Supervised Learning
• Hierarchical Reinforcement Learning Framework towards Multi-agent Navigation
• RRT* Combined with GVO for Real-time Nonholonomic Robot Navigation in Dynamic Environment
• Multi Pseudo Q-learning Based Deterministic Policy Gradient for Tracking Control of Autonomous Underwater Vehicles
• Point Cloud Based Reinforcement Learning for Sim-to-Real and Partial Observability in Visual Navigation