A Hysteretic Q-learning Coordination Framework for Emerging Mobility Systems in Smart Cities

Abstract

Connected and automated vehicles (CAVs) can alleviate traffic congestion, air pollution, and improve safety. In this paper, we provide a decentralized coordination framework for CAVs at a signal-free intersection to minimize travel time and improve fuel efficiency. We employ a simple yet powerful reinforcement learning approach, an off-policy temporal difference learning called Q-learning, enhanced with a coordination mechanism to address this problem. Then, we integrate a first-in-first-out queuing policy to improve the performance of our system. We demonstrate the efficacy of our proposed approach through simulation and comparison with the classical optimal control method based on Pontryagin's minimum principle.

arXiv comment8 pages, 5 figures, 2 tables

References in corpus (5)

• A Decentralized Energy-Optimal Control Framework for Connected Automated Vehicles at Signal-Free Intersections
• Optimal Time Trajectory and Coordination for Connected and Automated Vehicles
• Optimal Control of Connected and Automated Vehicles at Multiple Adjacent Intersections
• Zero-Shot Autonomous Vehicle Policy Transfer: From Simulation to Real-World via Adversarial Learning
• Simulation to Scaled City: Zero-Shot Policy Transfer for Traffic Control via Autonomous Vehicles

Cited by in corpus (5)

• A Research and Educational Robotic Testbed for Real-time Control of Emerging Mobility Systems: From Theory to Scaled Experiments
• Mechanism Design Theory in Control Engineering: A Tutorial and Overview of Applications in Communication, Power Grid, Transportation, and Security Systems
• Social Resource Allocation in a Mobility System with Connected and Automated Vehicles: A Mechanism Design Problem
• Sample and Communication-Efficient Decentralized Actor-Critic Algorithms with Finite-Time Analysis
• Multi-Agent Off-Policy TD Learning: Finite-Time Analysis with Near-Optimal Sample Complexity and Communication Complexity