Hybrid Car-Following Strategy based on Deep Deterministic Policy Gradient and Cooperative Adaptive Cruise Control
arXiv:2103.03796 · doi:10.1109/TASE.2021.3100709
Abstract
Deep deterministic policy gradient (DDPG)-based car-following strategy can break through the constraints of the differential equation model due to the ability of exploration on complex environments. However, the car-following performance of DDPG is usually degraded by unreasonable reward function design, insufficient training, and low sampling efficiency. In order to solve this kind of problem, a hybrid car-following strategy based on DDPG and cooperative adaptive cruise control (CACC) is proposed. First, the car-following process is modeled as the Markov decision process to calculate CACC and DDPG simultaneously at each frame. Given a current state, two actions are obtained from CACC and DDPG, respectively. Then, an optimal action, corresponding to the one offering a larger reward, is chosen as the output of the hybrid strategy. Meanwhile, a rule is designed to ensure that the change rate of acceleration is smaller than the desired value. Therefore, the proposed strategy not only guarantees the basic performance of car-following through CACC but also makes full use of the advantages of exploration on complex environments via DDPG. Finally, simulation results show that the car-following performance of the proposed strategy is improved compared with that of DDPG and CACC.
arXiv comment9 pages, 11 figures
References in corpus (6)
- Continuous control with deep reinforcement learning
- Dissipation of stop-and-go waves via control of autonomous vehicles: Field experiments
- Safe, Efficient, and Comfortable Velocity Control based on Reinforcement Learning for Autonomous Driving
- Traffic experiment reveals the nature of car-following
- Comparison of Deep Reinforcement Learning and Model Predictive Control for Adaptive Cruise Control
- Longitudinal Dynamic versus Kinematic Models for Car-Following Control Using Deep Reinforcement Learning