Flatland-RL : Multi-Agent Reinforcement Learning on Trains

Abstract

Efficient automated scheduling of trains remains a major challenge for modern railway systems. The underlying vehicle rescheduling problem (VRSP) has been a major focus of Operations Research (OR) since decades. Traditional approaches use complex simulators to study VRSP, where experimenting with a broad range of novel ideas is time consuming and has a huge computational overhead. In this paper, we introduce a two-dimensional simplified grid environment called "Flatland" that allows for faster experimentation. Flatland does not only reduce the complexity of the full physical simulation, but also provides an easy-to-use interface to test novel approaches for the VRSP, such as Reinforcement Learning (RL) and Imitation Learning (IL). In order to probe the potential of Machine Learning (ML) research on Flatland, we (1) ran a first series of RL and IL experiments and (2) design and executed a public Benchmark at NeurIPS 2020 to engage a large community of researchers to work on this problem. Our own experimental results, on the one hand, demonstrate that ML has potential in solving the VRSP on Flatland. On the other hand, we identify key topics that need further research. Overall, the Flatland environment has proven to be a robust and valuable framework to investigate the VRSP for railway networks. Our experiments provide a good starting point for further research and for the participants of the NeurIPS 2020 Flatland Benchmark. All of these efforts together have the potential to have a substantial impact on shaping the mobility of the future.

References in corpus (7)

• StarCraft II: A New Challenge for Reinforcement Learning
• PRIMAL: Pathfinding via Reinforcement and Imitation Multi-Agent Learning
• Deep Q-learning from Demonstrations
• Stabilizing Transformers for Reinforcement Learning
• The Multi-Agent Reinforcement Learning in MalmÖ (MARLÖ) Competition
• Predicting Tactical Solutions to Operational Planning Problems under Imperfect Information
• Predicting Tactical Solutions to Operational Planning Problems under Imperfect Information

Cited by in corpus (4)

• PRIMAL2: Pathfinding via Reinforcement and Imitation Multi-Agent Learning -- Lifelong
• Learning to run a Power Network Challenge: a Retrospective Analysis
• Flatland Competition 2020: MAPF and MARL for Efficient Train Coordination on a Grid World
• Towards robust and domain agnostic reinforcement learning competitions