Network-wide traffic signal control optimization using a multi-agent deep reinforcement learning

Abstract

Inefficient traffic control may cause numerous problems such as traffic congestion and energy waste. This paper proposes a novel multi-agent reinforcement learning method, named KS-DDPG (Knowledge Sharing Deep Deterministic Policy Gradient) to achieve optimal control by enhancing the cooperation between traffic signals. By introducing the knowledge-sharing enabled communication protocol, each agent can access to the collective representation of the traffic environment collected by all agents. The proposed method is evaluated through two experiments respectively using synthetic and real-world datasets. The comparison with state-of-the-art reinforcement learning-based and conventional transportation methods demonstrate the proposed KS-DDPG has significant efficiency in controlling large-scale transportation networks and coping with fluctuations in traffic flow. In addition, the introduced communication mechanism has also been proven to speed up the convergence of the model without significantly increasing the computational burden.

References in corpus (11)

• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• Congested Traffic States in Empirical Observations and Microscopic Simulations
• Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments
• Benchmarking Deep Reinforcement Learning for Continuous Control
• OpenAI Gym
• CoLight: Learning Network-level Cooperation for Traffic Signal Control
• Using a Deep Reinforcement Learning Agent for Traffic Signal Control
• Deep Deterministic Policy Gradient for Urban Traffic Light Control
• Optimal Passenger-Seeking Policies on E-hailing Platforms Using Markov Decision Process and Imitation Learning
• R-MADDPG for Partially Observable Environments and Limited Communication
• Partially Detected Intelligent Traffic Signal Control: Environmental Adaptation