Using a Deep Reinforcement Learning Agent for Traffic Signal Control

Abstract

Ensuring transportation systems are efficient is a priority for modern society. Technological advances have made it possible for transportation systems to collect large volumes of varied data on an unprecedented scale. We propose a traffic signal control system which takes advantage of this new, high quality data, with minimal abstraction compared to other proposed systems. We apply modern deep reinforcement learning methods to build a truly adaptive traffic signal control agent in the traffic microsimulator SUMO. We propose a new state space, the discrete traffic state encoding, which is information dense. The discrete traffic state encoding is used as input to a deep convolutional neural network, trained using Q-learning with experience replay. Our agent was compared against a one hidden layer neural network traffic signal control agent and reduces average cumulative delay by 82%, average queue length by 66% and average travel time by 20%.

Cited by in corpus (32)

• A Survey on Traffic Signal Control Methods
• Network-wide traffic signal control optimization using a multi-agent deep reinforcement learning
• Adaptive Traffic Signal Control: Deep Reinforcement Learning Algorithm with Experience Replay and Target Network
• Deep Deterministic Policy Gradient for Urban Traffic Light Control
• Applications of deep learning in traffic congestion detection, prediction and alleviation: A survey
• IG-RL: Inductive Graph Reinforcement Learning for Massive-Scale Traffic Signal Control
• Deep Reinforcement Learning Algorithm for Dynamic Pricing of Express Lanes with Multiple Access Locations
• Transformer-based Map Matching Model with Limited Ground-Truth Data using Transfer-Learning Approach
• Multi-Agent Reinforcement Learning Based on Representational Communication for Large-Scale Traffic Signal Control
• Autonomous Air Traffic Controller: A Deep Multi-Agent Reinforcement Learning Approach
• Integrating independent and centralized multi-agent reinforcement learning for traffic signal network optimization
• Diagnosing Reinforcement Learning for Traffic Signal Control
• Deep Reinforcement Learning and Transportation Research: A Comprehensive Review
• Using Reinforcement Learning with Partial Vehicle Detection for Intelligent Traffic Signal Control
• An Open-Source Framework for Adaptive Traffic Signal Control
• Large-scale traffic signal control using machine learning: some traffic flow considerations
• Deep Reinforcement Learning for Dynamic Urban Transportation Problems
• A Deep Reinforcement Learning Approach for Traffic Signal Control Optimization
• Optimizing Traffic Lights with Multi-agent Deep Reinforcement Learning and V2X communication
• Partially Detected Intelligent Traffic Signal Control: Environmental Adaptation
• Optimizing city-scale traffic through modeling observations of vehicle movements
• Multi-Agent Deep Reinforcement Learning for Large-scale Traffic Signal Control
• Intelligent Coordination among Multiple Traffic Intersections Using Multi-Agent Reinforcement Learning
• Dynamic Graph Configuration with Reinforcement Learning for Connected Autonomous Vehicle Trajectories
• Tensor-based Cooperative Control for Large Scale Multi-intersection Traffic Signal Using Deep Reinforcement Learning and Imitation Learning
• Using AI for Mitigating the Impact of Network Delay in Cloud-based Intelligent Traffic Signal Control
• Finding Appropriate Traffic Regulations via Graph Convolutional Networks
• Assessment of Reward Functions in Reinforcement Learning for Multi-Modal Urban Traffic Control under Real-World limitations
• Multi-intersection Traffic Optimisation: A Benchmark Dataset and a Strong Baseline
• A Simple Traffic Signal Control Using Queue Length Information
• Congested Urban Networks Tend to Be Insensitive to Signal Settings: Implications for Learning-Based Control
• A Methodology for the Development of RL-Based Adaptive Traffic Signal Controllers