Safe, Efficient, and Comfortable Velocity Control based on Reinforcement Learning for Autonomous Driving

Abstract

A model used for velocity control during car following was proposed based on deep reinforcement learning (RL). To fulfil the multi-objectives of car following, a reward function reflecting driving safety, efficiency, and comfort was constructed. With the reward function, the RL agent learns to control vehicle speed in a fashion that maximizes cumulative rewards, through trials and errors in the simulation environment. A total of 1,341 car-following events extracted from the Next Generation Simulation (NGSIM) dataset were used to train the model. Car-following behavior produced by the model were compared with that observed in the empirical NGSIM data, to demonstrate the model's ability to follow a lead vehicle safely, efficiently, and comfortably. Results show that the model demonstrates the capability of safe, efficient, and comfortable velocity control in that it 1) has small percentages (8\%) of dangerous minimum time to collision values (\textless\ 5s) than human drivers in the NGSIM data (35\%); 2) can maintain efficient and safe headways in the range of 1s to 2s; and 3) can follow the lead vehicle comfortably with smooth acceleration. The results indicate that reinforcement learning methods could contribute to the development of autonomous driving systems.

Under the first-round revision for transportation research part c

References in corpus (4)

• Continuous control with deep reinforcement learning
• Congested Traffic States in Empirical Observations and Microscopic Simulations
• Deep Reinforcement Learning: An Overview
• Comparison of Deep Reinforcement Learning and Model Predictive Control for Adaptive Cruise Control

Cited by in corpus (24)

• FollowNet: A Comprehensive Benchmark for Car-Following Behavior Modeling
• HAIM-DRL: Enhanced Human-in-the-loop Reinforcement Learning for Safe and Efficient Autonomous Driving
• Modeling driver's evasive behavior during safety-critical lane changes:Two-dimensional time-to-collision and deep reinforcement learning
• Hybrid Car-Following Strategy based on Deep Deterministic Policy Gradient and Cooperative Adaptive Cruise Control
• BehAVExplor: Behavior Diversity Guided Testing for Autonomous Driving Systems
• Autonomous Platoon Control with Integrated Deep Reinforcement Learning and Dynamic Programming
• Reliably-stabilizing piecewise-affine neural network controllers
• Physics-Augmented Data-EnablEd Predictive Control for Eco-driving of Mixed Traffic Considering Diverse Human Behaviors
• Adversarial Safety-Critical Scenario Generation using Naturalistic Human Driving Priors
• Safety in Traffic Management Systems: A Comprehensive Survey
• Formulation and validation of a car-following model based on deep reinforcement learning
• Deep Reinforcement Learning and Transportation Research: A Comprehensive Review
• Using Empirical Trajectory Data to Design Connected Autonomous Vehicle Controllers for Traffic Stabilization
• KARB Solution: Compliance to Quality by Rule Based Benchmarking
• SAINT-ACC: Safety-Aware Intelligent Adaptive Cruise Control for Autonomous Vehicles Using Deep Reinforcement Learning
• Vision-based DRL Autonomous Driving Agent with Sim2Real Transfer
• Traffic Co-Simulation Framework Empowered by Infrastructure Camera Sensing and Reinforcement Learning
• AVDDPG: Federated reinforcement learning applied to autonomous platoon control
• Dampen the Stop-and-Go Traffic with Connected and Automated Vehicles -- A Deep Reinforcement Learning Approach
• KI-PMF: Knowledge Integrated Plausible Motion Forecasting
• Towards Developing Socially Compliant Automated Vehicles: Advances, Expert Insights, and A Conceptual Framework
• Communication-Efficient Zeroth-Order Distributed Online Optimization: Algorithm, Theory, and Applications
• Network traffic instability in a two-ring system with automated driving and cooperative merging
• Shared learning of powertrain control policies for vehicle fleets