Variational End-to-End Navigation and Localization

Abstract

Deep learning has revolutionized the ability to learn "end-to-end" autonomous vehicle control directly from raw sensory data. While there have been recent extensions to handle forms of navigation instruction, these works are unable to capture the full distribution of possible actions that could be taken and to reason about localization of the robot within the environment. In this paper, we extend end-to-end driving networks with the ability to perform point-to-point navigation as well as probabilistic localization using only noisy GPS data. We define a novel variational network capable of learning from raw camera data of the environment as well as higher level roadmaps to predict (1) a full probability distribution over the possible control commands; and (2) a deterministic control command capable of navigating on the route specified within the map. Additionally, we formulate how our model can be used to localize the robot according to correspondences between the map and the observed visual road topology, inspired by the rough localization that human drivers can perform. We test our algorithms on real-world driving data that the vehicle has never driven through before, and integrate our point-to-point navigation algorithms onboard a full-scale autonomous vehicle for real-time performance. Our localization algorithm is also evaluated over a new set of roads and intersections to demonstrates rough pose localization even in situations without any GPS prior.

Published in IEEE International Conference on Robotics and Automation (ICRA) 2019. Best Paper Award Finalist

References in corpus (3)

• Safe, Multi-Agent, Reinforcement Learning for Autonomous Driving
• Intention-Net: Integrating Planning and Deep Learning for Goal-Directed Autonomous Navigation
• Uncertainty-Aware Driver Trajectory Prediction at Urban Intersections

Cited by in corpus (12)

• A Survey of End-to-End Driving: Architectures and Training Methods
• Learning to Walk in the Real World with Minimal Human Effort
• Controlling Steering Angle for Cooperative Self-driving Vehicles utilizing CNN and LSTM-based Deep Networks
• Probabilistic End-to-End Vehicle Navigation in Complex Dynamic Environments with Multimodal Sensor Fusion
• Combining Optimal Control and Learning for Visual Navigation in Novel Environments
• Deep Evidential Regression
• Uncertainty-Aware Driver Trajectory Prediction at Urban Intersections
• Towards navigation without precise localization: Weakly supervised learning of goal-directed navigation cost map
• You Are Here: Geolocation by Embedding Maps and Images
• DeepGoal: Learning to Drive with driving intention from Human Control Demonstration
• Failing with Grace: Learning Neural Network Controllers that are Boundedly Unsafe
• Learning Navigation by Visual Localization and Trajectory Prediction