Arena: a toolkit for Multi-Agent Reinforcement Learning
arXiv:1907.09467
Abstract
We introduce Arena, a toolkit for multi-agent reinforcement learning (MARL) research. In MARL, it usually requires customizing observations, rewards and actions for each agent, changing cooperative-competitive agent-interaction, and playing with/against a third-party agent, etc. We provide a novel modular design, called Interface, for manipulating such routines in essentially two ways: 1) Different interfaces can be concatenated and combined, which extends the OpenAI Gym Wrappers concept to MARL scenarios. 2) During MARL training or testing, interfaces can be embedded in either wrapped OpenAI Gym compatible Environments or raw environment compatible Agents. We offer off-the-shelf interfaces for several popular MARL platforms, including StarCraft II, Pommerman, ViZDoom, Soccer, etc. The interfaces effectively support self-play RL and cooperative-competitive hybrid MARL. Also, Arena can be conveniently extended to your own favorite MARL platform.
21 pages, 14 figures
References in corpus (9)
- Human-level performance in first-person multiplayer games with population-based deep reinforcement learning
- StarCraft II: A New Challenge for Reinforcement Learning
- DeepMind Control Suite
- Counterfactual Multi-Agent Policy Gradients
- Emergent Complexity via Multi-Agent Competition
- ViZDoom Competitions: Playing Doom from Pixels
- Gotta Learn Fast: A New Benchmark for Generalization in RL
- Learning to Act by Predicting the Future
- TStarBots: Defeating the Cheating Level Builtin AI in StarCraft II in the Full Game