Emergent Road Rules In Multi-Agent Driving Environments

Abstract

For autonomous vehicles to safely share the road with human drivers, autonomous vehicles must abide by specific "road rules" that human drivers have agreed to follow. "Road rules" include rules that drivers are required to follow by law -- such as the requirement that vehicles stop at red lights -- as well as more subtle social rules -- such as the implicit designation of fast lanes on the highway. In this paper, we provide empirical evidence that suggests that -- instead of hard-coding road rules into self-driving algorithms -- a scalable alternative may be to design multi-agent environments in which road rules emerge as optimal solutions to the problem of maximizing traffic flow. We analyze what ingredients in driving environments cause the emergence of these road rules and find that two crucial factors are noisy perception and agents' spatial density. We provide qualitative and quantitative evidence of the emergence of seven social driving behaviors, ranging from obeying traffic signals to following lanes, all of which emerge from training agents to drive quickly to destinations without colliding. Our results add empirical support for the social road rules that countries worldwide have agreed on for safe, efficient driving.

arXiv commentInternational Conference on Learning Representations (2021)

References in corpus (13)

• Playing Atari with Deep Reinforcement Learning
• End to End Learning for Self-Driving Cars
• High-Dimensional Continuous Control Using Generalized Advantage Estimation
• Learning to Communicate with Deep Multi-Agent Reinforcement Learning
• A Survey and Critique of Multiagent Deep Reinforcement Learning
• Horovod: fast and easy distributed deep learning in TensorFlow
• Emergent Tool Use From Multi-Agent Autocurricula
• Social Influence as Intrinsic Motivation for Multi-Agent Deep Reinforcement Learning
• A Survey of Learning in Multiagent Environments: Dealing with Non-Stationarity
• SMARTS: Scalable Multi-Agent Reinforcement Learning Training School for Autonomous Driving
• ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst
• Emergent Coordination Through Competition
• Autocurricula and the Emergence of Innovation from Social Interaction: A Manifesto for Multi-Agent Intelligence Research