SMARTS: Scalable Multi-Agent Reinforcement Learning Training School for Autonomous Driving

Abstract

Multi-agent interaction is a fundamental aspect of autonomous driving in the real world. Despite more than a decade of research and development, the problem of how to competently interact with diverse road users in diverse scenarios remains largely unsolved. Learning methods have much to offer towards solving this problem. But they require a realistic multi-agent simulator that generates diverse and competent driving interactions. To meet this need, we develop a dedicated simulation platform called SMARTS (Scalable Multi-Agent RL Training School). SMARTS supports the training, accumulation, and use of diverse behavior models of road users. These are in turn used to create increasingly more realistic and diverse interactions that enable deeper and broader research on multi-agent interaction. In this paper, we describe the design goals of SMARTS, explain its basic architecture and its key features, and illustrate its use through concrete multi-agent experiments on interactive scenarios. We open-source the SMARTS platform and the associated benchmark tasks and evaluation metrics to encourage and empower research on multi-agent learning for autonomous driving. Our code is available at https://github.com/huawei-noah/SMARTS.

arXiv comment20 pages, 11 figures. Paper accepted to CoRL 2020

References in corpus (16)

• Playing Atari with Deep Reinforcement Learning
• Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm
• Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments
• StarCraft II: A New Challenge for Reinforcement Learning
• Counterfactual Multi-Agent Policy Gradients
• Safe, Multi-Agent, Reinforcement Learning for Autonomous Driving
• CARLA: An Open Urban Driving Simulator
• Learning Multiagent Communication with Backpropagation
• Multiagent Bidirectionally-Coordinated Nets: Emergence of Human-level Coordination in Learning to Play StarCraft Combat Games
• QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learning
• Simulated Car Racing Championship: Competition Software Manual
• MAVEN: Multi-Agent Variational Exploration
• R-MADDPG for Partially Observable Environments and Limited Communication
• Probabilistic Recursive Reasoning for Multi-Agent Reinforcement Learning
• MADRaS : Multi Agent Driving Simulator
• A Regularized Opponent Model with Maximum Entropy Objective

Cited by in corpus (25)

• Stepwise Goal-Driven Networks for Trajectory Prediction
• Reinforcement Learning for Ridesharing: An Extended Survey
• FusionPlanner: A Multi-task Motion Planner for Mining Trucks via Multi-sensor Fusion
• Rethinking the Implementation Tricks and Monotonicity Constraint in Cooperative Multi-Agent Reinforcement Learning
• Settling the Variance of Multi-Agent Policy Gradients
• NAST: Non-Autoregressive Spatial-Temporal Transformer for Time Series Forecasting
• Rethinking Closed-loop Training for Autonomous Driving
• S4TP: Social-Suitable and Safety-Sensitive Trajectory Planning for Autonomous Vehicles
• Multi-Agent Constrained Policy Optimisation
• CARLA Real Traffic Scenarios -- novel training ground and benchmark for autonomous driving
• Efficient Deep Reinforcement Learning with Imitative Expert Priors for Autonomous Driving
• Learning in Nonzero-Sum Stochastic Games with Potentials
• Diverse Auto-Curriculum is Critical for Successful Real-World Multiagent Learning Systems
• Generative Adversarial Imitation Learning for End-to-End Autonomous Driving on Urban Environments
• Improving the Generalization of End-to-End Driving through Procedural Generation
• Deception in Social Learning: A Multi-Agent Reinforcement Learning Perspective
• Emergent Road Rules In Multi-Agent Driving Environments
• A Game-Theoretic Approach to Multi-Agent Trust Region Optimization
• RITA: Boost Driving Simulators with Realistic Interactive Traffic Flow
• LINDA: Multi-Agent Local Information Decomposition for Awareness of Teammates
• Revisiting the Characteristics of Stochastic Gradient Noise and Dynamics
• An End-to-End Collaborative Learning Approach for Connected Autonomous Vehicles in Occluded Scenarios
• Dream to Explore: Adaptive Simulations for Autonomous Systems
• DriverGym: Democratising Reinforcement Learning for Autonomous Driving
• Towards a Very Large Scale Traffic Simulator for Multi-Agent Reinforcement Learning Testbeds