Explaining Image Classifiers using Statistical Fault Localization

Abstract

The black-box nature of deep neural networks (DNNs) makes it impossible to understand why a particular output is produced, creating demand for "Explainable AI". In this paper, we show that statistical fault localization (SFL) techniques from software engineering deliver high quality explanations of the outputs of DNNs, where we define an explanation as a minimal subset of features sufficient for making the same decision as for the original input. We present an algorithm and a tool called DeepCover, which synthesizes a ranking of the features of the inputs using SFL and constructs explanations for the decisions of the DNN based on this ranking. We compare explanations produced by DeepCover with those of the state-of-the-art tools GradCAM, LIME, SHAP, RISE and Extremal and show that explanations generated by DeepCover are consistently better across a broad set of experiments. On a benchmark set with known ground truth, DeepCover achieves 76.7% accuracy, which is 6% better than the second best Extremal.

References in corpus (11)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Sanity Checks for Saliency Maps
• Guiding Deep Learning System Testing using Surprise Adequacy
• RISE: Randomized Input Sampling for Explanation of Black-box Models
• Automated Directed Fairness Testing
• Defining Explanation in Probabilistic Systems
• Combinatorial Testing for Deep Learning Systems
• Symbolic Execution for Deep Neural Networks
• Detecting Adversarial Samples for Deep Neural Networks through Mutation Testing
• DeepMutation: Mutation Testing of Deep Learning Systems
• DeepFault: Fault Localization for Deep Neural Networks