Scalable End-to-End Autonomous Vehicle Testing via Rare-event Simulation

Abstract

While recent developments in autonomous vehicle (AV) technology highlight substantial progress, we lack tools for rigorous and scalable testing. Real-world testing, the $\text{de facto}$ evaluation environment, places the public in danger, and, due to the rare nature of accidents, will require billions of miles in order to statistically validate performance claims. We implement a simulation framework that can test an entire modern autonomous driving system, including, in particular, systems that employ deep-learning perception and control algorithms. Using adaptive importance-sampling methods to accelerate rare-event probability evaluation, we estimate the probability of an accident under a base distribution governing standard traffic behavior. We demonstrate our framework on a highway scenario, accelerating system evaluation by $2$-$20$ times over naive Monte Carlo sampling methods and $10$-$300\mathsf{P}$ times (where $\mathsf{P}$ is the number of processors) over real-world testing.

NeurIPS 2018

References in corpus (10)

• End to End Learning for Self-Driving Cars
• The social dilemma of autonomous vehicles
• Weight Uncertainty in Neural Networks
• Deep Exploration via Bootstrapped DQN
• On a Formal Model of Safe and Scalable Self-driving Cars
• Generative Adversarial Imitation Learning
• CARLA: An Open Urban Driving Simulator
• Provable Bounds for Learning Some Deep Representations
• Spectrally-normalized margin bounds for neural networks
• Self-Driving Vehicle Verification Towards a Benchmark

Cited by in corpus (4)

• I Know You Can't See Me: Dynamic Occlusion-Aware Safety Validation of Strategic Planners for Autonomous Vehicles Using Hypergames
• Uncertainty Quantification and Confidence Intervals for Naive Rare-Event Estimators
• GLADAS: Gesture Learning for Advanced Driver Assistance Systems
• Testing a Battery Management System via Criticality-based Rare Event Simulation