Model-free Deep Reinforcement Learning for Urban Autonomous Driving

Abstract

Urban autonomous driving decision making is challenging due to complex road geometry and multi-agent interactions. Current decision making methods are mostly manually designing the driving policy, which might result in sub-optimal solutions and is expensive to develop, generalize and maintain at scale. On the other hand, with reinforcement learning (RL), a policy can be learned and improved automatically without any manual designs. However, current RL methods generally do not work well on complex urban scenarios. In this paper, we propose a framework to enable model-free deep reinforcement learning in challenging urban autonomous driving scenarios. We design a specific input representation and use visual encoding to capture the low-dimensional latent states. Several state-of-the-art model-free deep RL algorithms are implemented into our framework, with several tricks to improve their performance. We evaluate our method in a challenging roundabout task with dense surrounding vehicles in a high-definition driving simulator. The result shows that our method can solve the task well and is significantly better than the baseline.

arXiv comment7 pages, 6 figures

References in corpus (13)

• Adam: A Method for Stochastic Optimization
• Continuous control with deep reinforcement learning
• Playing Atari with Deep Reinforcement Learning
• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
• End to End Learning for Self-Driving Cars
• Trust Region Policy Optimization
• Addressing Function Approximation Error in Actor-Critic Methods
• Soft Actor-Critic Algorithms and Applications
• Tutorial on Variational Autoencoders
• CARLA: An Open Urban Driving Simulator
• Explaining How a Deep Neural Network Trained with End-to-End Learning Steers a Car
• Uncertainty-aware Short-term Motion Prediction of Traffic Actors for Autonomous Driving
• Deep Imitation Learning for Autonomous Driving in Generic Urban Scenarios with Enhanced Safety

Cited by in corpus (4)

• A Review of Deep Reinforcement Learning for Smart Building Energy Management
• A DRL-based Multiagent Cooperative Control Framework for CAV Networks: a Graphic Convolution Q Network
• Fully Bayesian Recurrent Neural Networks for Safe Reinforcement Learning
• Congested Urban Networks Tend to Be Insensitive to Signal Settings: Implications for Learning-Based Control