Causal Reasoning in Software Quality Assurance: A Systematic Review

Abstract

Context: Software Quality Assurance (SQA) is a fundamental part of software engineering to ensure stakeholders that software products work as expected after release in operation. Machine Learning (ML) has proven to be able to boost SQA activities and contribute to the development of quality software systems. In this context, Causal Reasoning is gaining increasing interest as a methodology to go beyond a purely data-driven approach by exploiting the use of causality for more effective SQA strategies. Objective: Provide a broad and detailed overview of the use of causal reasoning for SQA activities, in order to support researchers to access this research field, identifying room for application, main challenges and research opportunities. Methods: A systematic review of the scientific literature on causal reasoning for SQA. The study has found, classified, and analyzed 86 articles, according to established guidelines for software engineering secondary studies. Results: Results highlight the primary areas within SQA where causal reasoning has been applied, the predominant methodologies used, and the level of maturity of the proposed solutions. Fault localization is the activity where causal reasoning is more exploited, especially in the web services/microservices domain, but other tasks like testing are rapidly gaining popularity. Both causal inference and causal discovery are exploited, with the Pearl's graphical formulation of causality being preferred, likely due to its intuitiveness. Tools to favour their application are appearing at a fast pace - most of them after 2021. Conclusions: The findings show that causal reasoning is a valuable means for SQA tasks with respect to multiple quality attributes, especially during V&V, evolution and maintenance to ensure reliability, while it is not yet fully exploited for phases like ...

Accepted to Information and Software Technology journal

References in corpus (18)

• Auto-Encoding Variational Bayes
• Recursive Partitioning for Heterogeneous Causal Effects
• Granger causality and transfer entropy are equivalent for Gaussian variables
• Learning high-dimensional directed acyclic graphs with latent and selection variables
• Causal Discovery with Reinforcement Learning
• CausalRCA: Causal Inference based Precise Fine-grained Root Cause Localization for Microservice Applications
• Causal Machine Learning: A Survey and Open Problems
• Joint Causal Inference from Multiple Contexts
• DoWhy-GCM: An extension of DoWhy for causal inference in graphical causal models
• Applications of statistical causal inference in software engineering
• Causal Discovery Toolbox: Uncover causal relationships in Python
• Causal-learn: Causal Discovery in Python
• Causal structure based root cause analysis of outliers
• Inforence: Effective Fault Localization Based on Information-Theoretic Analysis and Statistical Causal Inference
• Towards Causal Analysis of Empirical Software Engineering Data: The Impact of Programming Languages on Coding Competitions
• CaRE: Finding Root Causes of Configuration Issues in Highly-Configurable Robots
• A Survey on Causal Reinforcement Learning
• An empirical study of Linespots: A novel past-fault algorithm