MAPPER: Multi-Agent Path Planning with Evolutionary Reinforcement Learning in Mixed Dynamic Environments

Abstract

Multi-agent navigation in dynamic environments is of great industrial value when deploying a large scale fleet of robot to real-world applications. This paper proposes a decentralized partially observable multi-agent path planning with evolutionary reinforcement learning (MAPPER) method to learn an effective local planning policy in mixed dynamic environments. Reinforcement learning-based methods usually suffer performance degradation on long-horizon tasks with goal-conditioned sparse rewards, so we decompose the long-range navigation task into many easier sub-tasks under the guidance of a global planner, which increases agents' performance in large environments. Moreover, most existing multi-agent planning approaches assume either perfect information of the surrounding environment or homogeneity of nearby dynamic agents, which may not hold in practice. Our approach models dynamic obstacles' behavior with an image-based representation and trains a policy in mixed dynamic environments without homogeneity assumption. To ensure multi-agent training stability and performance, we propose an evolutionary training approach that can be easily scaled to large and complex environments. Experiments show that MAPPER is able to achieve higher success rates and more stable performance when exposed to a large number of non-cooperative dynamic obstacles compared with traditional reaction-based planner LRA* and the state-of-the-art learning-based method.

6 pages, accepted at the 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2020)

References in corpus (5)

• Playing Atari with Deep Reinforcement Learning
• From Perception to Decision: A Data-driven Approach to End-to-end Motion Planning for Autonomous Ground Robots
• PRIMAL: Pathfinding via Reinforcement and Imitation Multi-Agent Learning
• Baidu Apollo EM Motion Planner
• Intrinsically Motivated Goal Exploration Processes with Automatic Curriculum Learning

Cited by in corpus (5)

• Decentralized Spatial-Temporal Trajectory Planning for Multicopter Swarms
• Constrained Model-based Reinforcement Learning with Robust Cross-Entropy Method
• EGO-Swarm: A Fully Autonomous and Decentralized Quadrotor Swarm System in Cluttered Environments
• Multi-agent navigation based on deep reinforcement learning and traditional pathfinding algorithm
• Subdimensional Expansion Using Attention-Based Learning For Multi-Agent Path Finding