Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Tasks
arXiv:2006.07869
Abstract
Multi-agent deep reinforcement learning (MARL) suffers from a lack of commonly-used evaluation tasks and criteria, making comparisons between approaches difficult. In this work, we provide a systematic evaluation and comparison of three different classes of MARL algorithms (independent learning, centralised multi-agent policy gradient, value decomposition) in a diverse range of cooperative multi-agent learning tasks. Our experiments serve as a reference for the expected performance of algorithms across different learning tasks, and we provide insights regarding the effectiveness of different learning approaches. We open-source EPyMARL, which extends the PyMARL codebase to include additional algorithms and allow for flexible configuration of algorithm implementation details such as parameter sharing. Finally, we open-source two environments for multi-agent research which focus on coordination under sparse rewards.
Published in 35th Conference on Neural Information Processing Systems (NeurIPS 2021) Track on Datasets and Benchmarks
References in corpus (9)
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
- StarCraft II: A New Challenge for Reinforcement Learning
- A Survey and Critique of Multiagent Deep Reinforcement Learning
- Learning a Generic Value-Selection Heuristic Inside a Constraint Programming Solver
- Social Influence as Intrinsic Motivation for Multi-Agent Deep Reinforcement Learning
- Benchmarking Model-Based Reinforcement Learning
- The Hanabi Challenge: A New Frontier for AI Research
- Dealing with Non-Stationarity in Multi-Agent Deep Reinforcement Learning