Automated Lane Change Strategy using Proximal Policy Optimization-based Deep Reinforcement Learning

Abstract

Lane-change maneuvers are commonly executed by drivers to follow a certain routing plan, overtake a slower vehicle, adapt to a merging lane ahead, etc. However, improper lane change behaviors can be a major cause of traffic flow disruptions and even crashes. While many rule-based methods have been proposed to solve lane change problems for autonomous driving, they tend to exhibit limited performance due to the uncertainty and complexity of the driving environment. Machine learning-based methods offer an alternative approach, as Deep reinforcement learning (DRL) has shown promising success in many application domains including robotic manipulation, navigation, and playing video games. However, applying DRL to autonomous driving still faces many practical challenges in terms of slow learning rates, sample inefficiency, and safety concerns. In this study, we propose an automated lane change strategy using proximal policy optimization-based deep reinforcement learning, which shows great advantages in learning efficiency while still maintaining stable performance. The trained agent is able to learn a smooth, safe, and efficient driving policy to make lane-change decisions (i.e. when and how) in a challenging situation such as dense traffic scenarios. The effectiveness of the proposed policy is validated by using metrics of task success rate and collision rate. The simulation results demonstrate the lane change maneuvers can be efficiently learned and executed in a safe, smooth, and efficient manner.

References in corpus (8)

• Playing Atari with Deep Reinforcement Learning
• Congested Traffic States in Empirical Observations and Microscopic Simulations
• End to End Learning for Self-Driving Cars
• Trust Region Policy Optimization
• Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization
• Trial without Error: Towards Safe Reinforcement Learning via Human Intervention
• Cooperative Lane Changing via Deep Reinforcement Learning
• Quadratic Q-network for Learning Continuous Control for Autonomous Vehicles

Cited by in corpus (6)

• Deep Reinforcement Learning and Transportation Research: A Comprehensive Review
• Decision-making for Autonomous Vehicles on Highway: Deep Reinforcement Learning with Continuous Action Horizon
• SAINT-ACC: Safety-Aware Intelligent Adaptive Cruise Control for Autonomous Vehicles Using Deep Reinforcement Learning
• A Survey of Deep Reinforcement Learning Algorithms for Motion Planning and Control of Autonomous Vehicles
• A Comparative Analysis of Deep Reinforcement Learning-enabled Freeway Decision-making for Automated Vehicles
• A Self-adaptive LSAC-PID Approach based on Lyapunov Reward Shaping for Mobile Robots