Accelerated Evaluation of Automated Vehicles Safety in Lane Change Scenarios Based on Importance Sampling Techniques
arXiv:1605.04965 · doi:10.1109/TITS.2016.2582208
Abstract
Automated vehicles (AVs) must be evaluated thoroughly before their release and deployment. A widely-used evaluation approach is the Naturalistic-Field Operational Test (N-FOT), which tests prototype vehicles directly on the public roads. Due to the low exposure to safety-critical scenarios, N-FOTs are time-consuming and expensive to conduct. In this paper, we propose an accelerated evaluation approach for AVs. The results can be used to generate motions of the primary other vehicles to accelerate the verification of AVs in simulations and controlled experiments. Frontal collision due to unsafe cut-ins is the target crash type of this paper. Human-controlled vehicles making unsafe lane changes are modeled as the primary disturbance to AVs based on data collected by the University of Michigan Safety Pilot Model Deployment Program. The cut-in scenarios are generated based on skewed statistics of collected human driver behaviors, which generate risky testing scenarios while preserving the statistical information so that the safety benefits of AVs in non-accelerated cases can be accurately estimated. The Cross Entropy method is used to recursively search for the optimal skewing parameters. The frequencies of occurrence of conflicts, crashes and injuries are estimated for a modeled automated vehicle, and the achieved accelerated rate is around 2,000 to 20,000. In other words, in the accelerated simulations, driving for 1,000 miles will expose the AV with challenging scenarios that will take about 2 to 20 million miles of real-world driving to encounter. This technique thus has the potential to reduce greatly the development and validation time for AVs.
submitted to IEEE ITSC
References in corpus (2)
Cited by in corpus (47)
- Accelerated Evaluation of Automated Vehicles Safety in Lane Change Scenarios Based on Importance Sampling Techniques
- Testing Scenario Library Generation for Connected and Automated Vehicles, Part I: Methodology
- Milestones in Autonomous Driving and Intelligent Vehicles Part I: Control, Computing System Design, Communication, HD Map, Testing, and Human Behaviors
- Testing Scenario Library Generation for Connected and Automated Vehicles, Part II: Case Studies
- A Survey of Algorithms for Black-Box Safety Validation of Cyber-Physical Systems
- Testing Scenario Library Generation for Connected and Automated Vehicles: An Adaptive Framework
- Scenario Parameter Generation Method and Scenario Representativeness Metric for Scenario-Based Assessment of Automated Vehicles
- LawBreaker: An Approach for Specifying Traffic Laws and Fuzzing Autonomous Vehicles
- Speed trajectory planning at signalized intersections using sequential convex optimization
- A Game-Theoretic Framework for Autonomous Vehicles Velocity Control: Bridging Microscopic Differential Games and Macroscopic Mean Field Games
- A Survey on Scenario-Based Testing for Automated Driving Systems in High-Fidelity Simulation
- LLM-attacker: Enhancing Closed-loop Adversarial Scenario Generation for Autonomous Driving with Large Language Models
- Finding Critical Scenarios for Automated Driving Systems: A Systematic Literature Review
- Safety of the Intended Driving Behavior Using Rulebooks
- Statistical Characteristics of Driver Acceleration Behavior and Its Probability Model
- Game-theoretic Modeling of Traffic in Unsignalized Intersection Network for Autonomous Vehicle Control Verification and Validation
- Distributionally Consistent Simulation of Naturalistic Driving Environment for Autonomous Vehicle Testing
- A behavior driven approach for sampling rare event situations for autonomous vehicles
- ACAV: A Framework for Automatic Causality Analysis in Autonomous Vehicle Accident Recordings
- Adversarial Evaluation of Autonomous Vehicles in Lane-Change Scenarios
- Adaptive Safety Evaluation for Connected and Automated Vehicles with Sparse Control Variates
- Learning to Collide: An Adaptive Safety-Critical Scenarios Generating Method
- I Know You Can't See Me: Dynamic Occlusion-Aware Safety Validation of Strategic Planners for Autonomous Vehicles Using Hypergames
- On the Adversarial Scenario-based Safety Testing of Robots: the Comparability and Optimal Aggressiveness
- Evaluation of Automated Vehicles Encountering Pedestrians at Unsignalized Crossings
- On Safety Testing, Validation, and Characterization with Scenario-Sampling: A Case Study of Legged Robots
- Few-Shot Scenario Testing for Autonomous Vehicles Based on Neighborhood Coverage and Similarity
- Real-world Troublemaker: A 5G Cloud-controlled Track Testing Framework for Automated Driving Systems in Safety-critical Interaction Scenarios
- Uncertainty Quantification and Confidence Intervals for Naive Rare-Event Estimators
- Modeling Multi-Vehicle Interaction Scenarios Using Gaussian Random Field
- Adaptive Stress Testing for Autonomous Vehicles
- A Gradual Takeover Strategy of the Active Safety System
- Big Data Generated by Connected and Automated Vehicles for Safety Monitoring, Assessment and Improvement, Final Report (Year 3)
- Transfer Importance Sampling -- How Testing Automated Vehicles in Multiple Test Setups Helps With the Bias-Variance Tradeoff
- An Interaction-aware Evaluation Method for Highly Automated Vehicles
- Efficient falsification approach for autonomous vehicle validation using a parameter optimisation technique based on reinforcement learning
- Evaluation Uncertainty in Data-Driven Self-Driving Testing
- Make Full Use of Testing Information: An Integrated Accelerated Testing and Evaluation Method for Autonomous Driving Systems
- Analysis of Unprotected Intersection Left-Turn Conflicts based on Naturalistic Driving Data
- Testing a Battery Management System via Criticality-based Rare Event Simulation
- Towards a Fatality-Aware Benchmark of Probabilistic Reaction Prediction in Highly Interactive Driving Scenarios
- Closed-Loop Policies for Operational Tests of Safety-Critical Systems
- Discovering Avoidable Planner Failures of Autonomous Vehicles using Counterfactual Analysis in Behaviorally Diverse Simulation
- Smart Black Box 2.0: Efficient High-bandwidth Driving Data Collection based on Video Anomalies
- Evaluation of Lane Departure Correction Systems Using a Stochastic Driver Model
- Coverage-Guided Road Selection and Prioritization for Efficient Testing in Autonomous Driving Systems
- Calibrating Over-Parametrized Simulation Models: A Framework via Eligibility Set