A DRL-based Multiagent Cooperative Control Framework for CAV Networks: a Graphic Convolution Q Network

Abstract

Connected Autonomous Vehicle (CAV) Network can be defined as a collection of CAVs operating at different locations on a multilane corridor, which provides a platform to facilitate the dissemination of operational information as well as control instructions. Cooperation is crucial in CAV operating systems since it can greatly enhance operation in terms of safety and mobility, and high-level cooperation between CAVs can be expected by jointly plan and control within CAV network. However, due to the highly dynamic and combinatory nature such as dynamic number of agents (CAVs) and exponentially growing joint action space in a multiagent driving task, achieving cooperative control is NP hard and cannot be governed by any simple rule-based methods. In addition, existing literature contains abundant information on autonomous driving's sensing technology and control logic but relatively little guidance on how to fuse the information acquired from collaborative sensing and build decision processor on top of fused information. In this paper, a novel Deep Reinforcement Learning (DRL) based approach combining Graphic Convolution Neural Network (GCN) and Deep Q Network (DQN), namely Graphic Convolution Q network (GCQ) is proposed as the information fusion module and decision processor. The proposed model can aggregate the information acquired from collaborative sensing and output safe and cooperative lane changing decisions for multiple CAVs so that individual intention can be satisfied even under a highly dynamic and partially observed mixed traffic. The proposed algorithm can be deployed on centralized control infrastructures such as road-side units (RSU) or cloud platforms to improve the CAV operation.

TRB 2021 Annual Meeting

References in corpus (10)

• Semi-Supervised Classification with Graph Convolutional Networks
• Playing Atari with Deep Reinforcement Learning
• A Survey and Critique of Multiagent Deep Reinforcement Learning
• DeepInf: Social Influence Prediction with Deep Learning
• Mean Field Multi-Agent Reinforcement Learning
• Measuring and Relieving the Over-smoothing Problem for Graph Neural Networks from the Topological View
• Model-free Deep Reinforcement Learning for Urban Autonomous Driving
• Leveraging Vehicle Connectivity and Autonomy to Stabilize Flow in Mixed Traffic Conditions: Accounting for Human-driven Vehicle Driver Behavioral Heterogeneity and Perception-reaction Time Delay
• Leveraging the Capabilities of Connected and Autonomous Vehicles and Multi-Agent Reinforcement Learning to Mitigate Highway Bottleneck Congestion
• Using Empirical Trajectory Data to Design Connected Autonomous Vehicle Controllers for Traffic Stabilization

Cited by in corpus (9)

• Graph-based Deep Learning for Communication Networks: A Survey
• Multi-agent Reinforcement Learning for Cooperative Lane Changing of Connected and Autonomous Vehicles in Mixed Traffic
• Deep Multi-agent Reinforcement Learning for Highway On-Ramp Merging in Mixed Traffic
• A Cooperative Control Framework for CAV Lane Change in a Mixed Traffic Environment
• Leveraging the Capabilities of Connected and Autonomous Vehicles and Multi-Agent Reinforcement Learning to Mitigate Highway Bottleneck Congestion
• Facilitating Connected Autonomous Vehicle Operations Using Space-weighted Information Fusion and Deep Reinforcement Learning Based Control
• Using Empirical Trajectory Data to Design Connected Autonomous Vehicle Controllers for Traffic Stabilization
• Decentralized Cooperative Lane Changing at Freeway Weaving Areas Using Multi-Agent Deep Reinforcement Learning
• An Overview on the Application of Graph Neural Networks in Wireless Networks