PettingZoo: Gym for Multi-Agent Reinforcement Learning

Abstract

This paper introduces the PettingZoo library and the accompanying Agent Environment Cycle ("AEC") games model. PettingZoo is a library of diverse sets of multi-agent environments with a universal, elegant Python API. PettingZoo was developed with the goal of accelerating research in Multi-Agent Reinforcement Learning ("MARL"), by making work more interchangeable, accessible and reproducible akin to what OpenAI's Gym library did for single-agent reinforcement learning. PettingZoo's API, while inheriting many features of Gym, is unique amongst MARL APIs in that it's based around the novel AEC games model. We argue, in part through case studies on major problems in popular MARL environments, that the popular game models are poor conceptual models of games commonly used in MARL and accordingly can promote confusing bugs that are hard to detect, and that the AEC games model addresses these problems.

References in corpus (13)

• Distributed Prioritized Experience Replay
• Multi-agent Reinforcement Learning in Sequential Social Dilemmas
• The Hanabi Challenge: A New Frontier for AI Research
• OpenSpiel: A Framework for Reinforcement Learning in Games
• Extending the OpenAI Gym for robotics: a toolkit for reinforcement learning using ROS and Gazebo
• Emergent Coordination Through Competition
• Inequity aversion improves cooperation in intertemporal social dilemmas
• MAgent: A Many-Agent Reinforcement Learning Platform for Artificial Collective Intelligence
• RLCard: A Toolkit for Reinforcement Learning in Card Games
• Revisiting Parameter Sharing in Multi-Agent Deep Reinforcement Learning
• SuperSuit: Simple Microwrappers for Reinforcement Learning Environments
• Multiplayer Support for the Arcade Learning Environment
• Agent Environment Cycle Games

Cited by in corpus (25)

• Game-Theoretic Multiagent Reinforcement Learning
• MO-MIX: Multi-Objective Multi-Agent Cooperative Decision-Making With Deep Reinforcement Learning
• Multi-agent Deep Reinforcement Learning for Distributed Load Restoration
• LibSignal: An Open Library for Traffic Signal Control
• DiNNO: Distributed Neural Network Optimization for Multi-Robot Collaborative Learning
• Poisoning Deep Reinforcement Learning Agents with In-Distribution Triggers
• Revisiting Parameter Sharing in Multi-Agent Deep Reinforcement Learning
• A reinforcement learning path planning approach for range-only underwater target localization with autonomous vehicles
• RFRL Gym: A Reinforcement Learning Testbed for Cognitive Radio Applications
• Canaries and Whistles: Resilient Drone Communication Networks with (or without) Deep Reinforcement Learning
• Collective Intelligence for Deep Learning: A Survey of Recent Developments
• On the Use and Misuse of Absorbing States in Multi-agent Reinforcement Learning
• Multiplayer Support for the Arcade Learning Environment
• Health-Informed Policy Gradients for Multi-Agent Reinforcement Learning
• Agent Environment Cycle Games
• Graph Attention Multi-Agent Fleet Autonomy for Advanced Air Mobility
• Foresight of Graph Reinforcement Learning Latent Permutations Learnt by Gumbel Sinkhorn Network
• DSSE: a drone swarm search environment
• A coevolutionary approach to deep multi-agent reinforcement learning
• Sim-Env: Decoupling OpenAI Gym Environments from Simulation Models
• Learning Altruistic Behaviours in Reinforcement Learning without External Rewards
• PowerGridworld: A Framework for Multi-Agent Reinforcement Learning in Power Systems
• PeersimGym: An Environment for Solving the Task Offloading Problem with Reinforcement Learning
• Greedy UnMixing for Q-Learning in Multi-Agent Reinforcement Learning
• Regularize! Don't Mix: Multi-Agent Reinforcement Learning without Explicit Centralized Structures