Diverse Auto-Curriculum is Critical for Successful Real-World Multiagent Learning Systems
arXiv:2102.07659
Abstract
Multiagent reinforcement learning (MARL) has achieved a remarkable amount of success in solving various types of video games. A cornerstone of this success is the auto-curriculum framework, which shapes the learning process by continually creating new challenging tasks for agents to adapt to, thereby facilitating the acquisition of new skills. In order to extend MARL methods to real-world domains outside of video games, we envision in this blue sky paper that maintaining a diversity-aware auto-curriculum is critical for successful MARL applications. Specifically, we argue that \emph{behavioural diversity} is a pivotal, yet under-explored, component for real-world multiagent learning systems, and that significant work remains in understanding how to design a diversity-aware auto-curriculum. We list four open challenges for auto-curriculum techniques, which we believe deserve more attention from this community. Towards validating our vision, we recommend modelling realistic interactive behaviours in autonomous driving as an important test bed, and recommend the SMARTS/ULTRA benchmark.
AAMAS 2021
References in corpus (9)
- Robots that can adapt like animals
- Mobility promotes and jeopardizes biodiversity in rock-paper-scissors games
- Illuminating search spaces by mapping elites
- INTERACTION Dataset: An INTERnational, Adversarial and Cooperative moTION Dataset in Interactive Driving Scenarios with Semantic Maps
- CARLA: An Open Urban Driving Simulator
- Challenges of Real-World Reinforcement Learning
- SMARTS: Scalable Multi-Agent Reinforcement Learning Training School for Autonomous Driving
- Autocurricula and the Emergence of Innovation from Social Interaction: A Manifesto for Multi-Agent Intelligence Research
- Multi-Agent Determinantal Q-Learning