CityFlow: A Multi-Agent Reinforcement Learning Environment for Large Scale City Traffic Scenario

Abstract

Traffic signal control is an emerging application scenario for reinforcement learning. Besides being as an important problem that affects people's daily life in commuting, traffic signal control poses its unique challenges for reinforcement learning in terms of adapting to dynamic traffic environment and coordinating thousands of agents including vehicles and pedestrians. A key factor in the success of modern reinforcement learning relies on a good simulator to generate a large number of data samples for learning. The most commonly used open-source traffic simulator SUMO is, however, not scalable to large road network and large traffic flow, which hinders the study of reinforcement learning on traffic scenarios. This motivates us to create a new traffic simulator CityFlow with fundamentally optimized data structures and efficient algorithms. CityFlow can support flexible definitions for road network and traffic flow based on synthetic and real-world data. It also provides user-friendly interface for reinforcement learning. Most importantly, CityFlow is more than twenty times faster than SUMO and is capable of supporting city-wide traffic simulation with an interactive render for monitoring. Besides traffic signal control, CityFlow could serve as the base for other transportation studies and can create new possibilities to test machine learning methods in the intelligent transportation domain.

WWW 2019 Demo Paper

Cited by in corpus (13)

• CoLight: Learning Network-level Cooperation for Traffic Signal Control
• Illumination-Adaptive Person Re-identification
• Choose Your Simulator Wisely: A Review on Open-source Simulators for Autonomous Driving
• BuildSenSys: Reusing Building Sensing Data for Traffic Prediction with Cross-domain Learning
• FusionPlanner: A Multi-task Motion Planner for Mining Trucks via Multi-sensor Fusion
• GeneraLight: Improving Environment Generalization of Traffic Signal Control via Meta Reinforcement Learning
• LibSignal: An Open Library for Traffic Signal Control
• DynSTGAT: Dynamic Spatial-Temporal Graph Attention Network for Traffic Signal Control
• Open-TI: Open Traffic Intelligence with Augmented Language Model
• Towards Multi-agent Reinforcement Learning based Traffic Signal Control through Spatio-temporal Hypergraphs
• Learning Traffic Signal Control via Genetic Programming
• Traffic Intersection Simulation Using Turning Movement Count Data in SUMO: A Case Study of Toronto Intersections
• Purpose in the Machine: Do Traffic Simulators Produce Distributionally Equivalent Outcomes for Reinforcement Learning Applications?