Toward Generating Sufficiently Valid Test Case Results: A Method for Systematically Assigning Test Cases to Test Bench Configurations in a Scenario-Based Test Approach for Automated Vehicles

Abstract

To successfully launch automated vehicles into the consumer market, there must be credible proof that the vehicles will operate safely. However, finding a method to validate the vehicles' safe operation is a challenging problem. While scenario-based test approaches seem to be possible solutions, they require execution of a large number of test cases. Several test benches, ranging from actual test vehicles to partly or fully simulated environments, are available to execute these test cases. Each test bench provides different elements, which in turn, have different parameters and parameter ranges. The composition of elements with their specific parameter values at a specific test bench that is used to execute a test case is referred to as a test bench configuration. However, selecting the most suitable test bench configuration is difficult. The selected test bench configuration determines whether the execution of a specific test case provides sufficiently valid test case results with respect to the intended purpose, for example, validating a vehicle's safe operation. The effective and efficient execution of a large number of test cases requires a method for systematically assigning test cases to the most suitable test bench configuration. Based on a proposed method for classifying test bench configurations, we propose and illustrate a method for systematically assigning test cases to test bench configurations in a scenario-based test approach for automated vehicles. This assignment method allows for the effective and efficient execution of a large number of test cases while generating sufficiently valid test case results.

Published in IEEE Access, vol. 10, pp. 6260-6285, 2022, doi: 10.1109/ACCESS.2022.3141198

References in corpus (3)

• Toward a Consistent Taxonomy for Scenario-Based Development and Test Approaches for Automated Vehicles: A Proposal for a Structuring Framework, a Basic Vocabulary, and its Application
• Toward Generating Sufficiently Valid Test Case Results: A Method for Systematically Assigning Test Cases to Test Bench Configurations in a Scenario-Based Test Approach for Automated Vehicles
• A Method for Classifying Test Bench Configurations in a Scenario-Based Test Approach for Automated Vehicles

Cited by in corpus (6)

• Toward Generating Sufficiently Valid Test Case Results: A Method for Systematically Assigning Test Cases to Test Bench Configurations in a Scenario-Based Test Approach for Automated Vehicles
• Toward Unsupervised Test Scenario Extraction for Automated Driving Systems from Urban Naturalistic Road Traffic Data
• Transfer Importance Sampling -- How Testing Automated Vehicles in Multiple Test Setups Helps With the Bias-Variance Tradeoff
• Methodology for Test Case Allocation based on a Formalized ODD
• Integration of an Agent Model into an Open Simulation Architecture for Scenario-Based Testing of Automated Vehicles
• A Concept for Semi-Automatic Configuration of Sufficiently Valid Simulation Setups for Automated Driving Systems