Diverse Auto-Curriculum is Critical for Successful Real-World Multiagent Learning Systems
arXiv:2102.07659
Abstract
Multiagent reinforcement learning (MARL) has achieved a remarkable amount of success in solving various types of video games. A cornerstone of this success is the auto-curriculum framework, which shapes the learning process by continually creating new challenging tasks for agents to adapt to, thereby facilitating the acquisition of new skills. In order to extend MARL methods to real-world domains outside of video games, we envision in this blue sky paper that maintaining a diversity-aware auto-curriculum is critical for successful MARL applications. Specifically, we argue that \emph{behavioural diversity} is a pivotal, yet under-explored, component for real-world multiagent learning systems, and that significant work remains in understanding how to design a diversity-aware auto-curriculum. We list four open challenges for auto-curriculum techniques, which we believe deserve more attention from this community. Towards validating our vision, we recommend modelling realistic interactive behaviours in autonomous driving as an important test bed, and recommend the SMARTS/ULTRA benchmark.
arXiv commentAAMAS 2021
References in corpus (32)
- Overcoming catastrophic forgetting in neural networks
- Playing Atari with Deep Reinforcement Learning
- Congested Traffic States in Empirical Observations and Microscopic Simulations
- Robots that can adapt like animals
- Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems
- Mobility promotes and jeopardizes biodiversity in rock-paper-scissors games
- Human-level performance in first-person multiplayer games with population-based deep reinforcement learning
- A Survey and Critique of Multiagent Deep Reinforcement Learning
- Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning
- Illuminating search spaces by mapping elites
- Emergent Tool Use From Multi-Agent Autocurricula
- Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review
- INTERACTION Dataset: An INTERnational, Adversarial and Cooperative moTION Dataset in Interactive Driving Scenarios with Semantic Maps
- CARLA: An Open Urban Driving Simulator
- Learning Dexterous In-Hand Manipulation
- Multiagent Bidirectionally-Coordinated Nets: Emergence of Human-level Coordination in Learning to Play StarCraft Combat Games
- Challenges of Real-World Reinforcement Learning
- The Hanabi Challenge: A New Frontier for AI Research
- Stochastic Neural Networks for Hierarchical Reinforcement Learning
- Flows and Decompositions of Games: Harmonic and Potential Games
- Game-Theoretic Multiagent Reinforcement Learning
- Sim-to-Real Robot Learning from Pixels with Progressive Nets
- SMARTS: Scalable Multi-Agent Reinforcement Learning Training School for Autonomous Driving
- Sim-to-Real Reinforcement Learning for Deformable Object Manipulation
- Emergent Coordination Through Competition
- Inequity aversion improves cooperation in intertemporal social dilemmas
- Autocurricula and the Emergence of Innovation from Social Interaction: A Manifesto for Multi-Agent Intelligence Research
- An empirical investigation of the challenges of real-world reinforcement learning
- Effective Diversity in Population Based Reinforcement Learning
- TStarBot-X: An Open-Sourced and Comprehensive Study for Efficient League Training in StarCraft II Full Game
- Real World Games Look Like Spinning Tops
- Multi-Agent Determinantal Q-Learning