A Survey of the Metrics, Uses, and Subjects of Diversity-Based Techniques in Software Testing
arXiv:2311.09714 · doi:10.1002/stvr.1914
Abstract
There has been a significant amount of interest regarding the use of diversity-based testing techniques in software testing over the past two decades. Diversity-based testing (DBT) technique uses similarity metrics to leverage the dissimilarity between software artefacts - such as requirements, abstract models, program structures, or inputs - in order to address a software testing problem. DBT techniques have been used to assist in finding solutions to several different types of problems including generating test cases, prioritising them, and reducing very large test suites. This paper is a systematic survey of DBT techniques that summarises the key aspects and trends of 144 papers that report the use of 70 different similarity metrics with 24 different types of software artefacts, which have been used by researchers to tackle 11 different types of software testing problems. We further present an analysis of the recent trends in DBT techniques and review the different application domains to which the techniques have been applied, giving an overview of the tools developed by researchers to do so. Finally, the paper identifies some DBT challenges that are potential topics for future work.
58 pages, 7 figures, 9 tables, and uses PRIMEarxiv.sty
References in corpus (18)
- Comprehensive Survey and Taxonomies of False Injection Attacks in Smart Grid: Attack Models, Targets, and Impacts
- BugsInPy: A Database of Existing Bugs in Python Programs to Enable Controlled Testing and Debugging Studies
- Black-Box Testing of Deep Neural Networks Through Test Case Diversity
- BehAVExplor: Behavior Diversity Guided Testing for Autonomous Driving Systems
- Diversity can be Transferred: Output Diversification for White- and Black-box Attacks
- SemMT: A Semantic-based Testing Approach for Machine Translation Systems
- DeepGD: A Multi-Objective Black-Box Test Selection Approach for Deep Neural Networks
- BeDivFuzz: Integrating Behavioral Diversity into Generator-based Fuzzing
- Black-box Safety Analysis and Retraining of DNNs based on Feature Extraction and Clustering
- Test case prioritization using test case diversification and fault-proneness estimations
- Two is Better Than One: Digital Siblings to Improve Autonomous Driving Testing
- Boundary Value Exploration for Software Analysis
- Simulator-based explanation and debugging of hazard-triggering events in DNN-based safety-critical systems
- A Comprehensive Empirical Evaluation of Generating Test Suites for Mobile Applications with Diversity
- Using mutation testing to measure behavioural test diversity
- Configuring Test Generators using Bug Reports: A Case Study of GCC Compiler and Csmith
- DeepRNG: Towards Deep Reinforcement Learning-Assisted Generative Testing of Software
- Feature Map Testing for Deep Neural Networks