Safe, Efficient, and Comfortable Velocity Control based on Reinforcement Learning for Autonomous Driving

Abstract

A model used for velocity control during car following was proposed based on deep reinforcement learning (RL). To fulfil the multi-objectives of car following, a reward function reflecting driving safety, efficiency, and comfort was constructed. With the reward function, the RL agent learns to control vehicle speed in a fashion that maximizes cumulative rewards, through trials and errors in the simulation environment. A total of 1,341 car-following events extracted from the Next Generation Simulation (NGSIM) dataset were used to train the model. Car-following behavior produced by the model were compared with that observed in the empirical NGSIM data, to demonstrate the model's ability to follow a lead vehicle safely, efficiently, and comfortably. Results show that the model demonstrates the capability of safe, efficient, and comfortable velocity control in that it 1) has small percentages (8\%) of dangerous minimum time to collision values (\textless\ 5s) than human drivers in the NGSIM data (35\%); 2) can maintain efficient and safe headways in the range of 1s to 2s; and 3) can follow the lead vehicle comfortably with smooth acceleration. The results indicate that reinforcement learning methods could contribute to the development of autonomous driving systems.

arXiv commentUnder the first-round revision for transportation research part c

References in corpus (4)

• Continuous control with deep reinforcement learning
• Congested Traffic States in Empirical Observations and Microscopic Simulations
• Deep Reinforcement Learning: An Overview
• Comparison of Deep Reinforcement Learning and Model Predictive Control for Adaptive Cruise Control

Cited by in corpus (24)

• FollowNet: A Comprehensive Benchmark for Car-Following Behavior Modeling
• HAIM-DRL: Enhanced Human-in-the-loop Reinforcement Learning for Safe and Efficient Autonomous Driving
• Modeling driver's evasive behavior during safety-critical lane changes:Two-dimensional time-to-collision and deep reinforcement learning
• Hybrid Car-Following Strategy based on Deep Deterministic Policy Gradient and Cooperative Adaptive Cruise Control
• BehAVExplor: Behavior Diversity Guided Testing for Autonomous Driving Systems
• Autonomous Platoon Control with Integrated Deep Reinforcement Learning and Dynamic Programming
• Reliably-stabilizing piecewise-affine neural network controllers
• Physics-Augmented Data-EnablEd Predictive Control for Eco-driving of Mixed Traffic Considering Diverse Human Behaviors
• Adversarial Safety-Critical Scenario Generation using Naturalistic Human Driving Priors
• Safety in Traffic Management Systems: A Comprehensive Survey
• Formulation and validation of a car-following model based on deep reinforcement learning
• Deep Reinforcement Learning and Transportation Research: A Comprehensive Review
• Using Empirical Trajectory Data to Design Connected Autonomous Vehicle Controllers for Traffic Stabilization
• KARB Solution: Compliance to Quality by Rule Based Benchmarking
• SAINT-ACC: Safety-Aware Intelligent Adaptive Cruise Control for Autonomous Vehicles Using Deep Reinforcement Learning
• Vision-based DRL Autonomous Driving Agent with Sim2Real Transfer
• Traffic Co-Simulation Framework Empowered by Infrastructure Camera Sensing and Reinforcement Learning
• AVDDPG: Federated reinforcement learning applied to autonomous platoon control
• Dampen the Stop-and-Go Traffic with Connected and Automated Vehicles -- A Deep Reinforcement Learning Approach
• KI-PMF: Knowledge Integrated Plausible Motion Forecasting
• Towards Developing Socially Compliant Automated Vehicles: Advances, Expert Insights, and A Conceptual Framework
• Communication-Efficient Zeroth-Order Distributed Online Optimization: Algorithm, Theory, and Applications
• Network traffic instability in a two-ring system with automated driving and cooperative merging
• Shared learning of powertrain control policies for vehicle fleets