On Completeness-aware Concept-Based Explanations in Deep Neural Networks

Abstract

Human explanations of high-level decisions are often expressed in terms of key concepts the decisions are based on. In this paper, we study such concept-based explainability for Deep Neural Networks (DNNs). First, we define the notion of completeness, which quantifies how sufficient a particular set of concepts is in explaining a model's prediction behavior based on the assumption that complete concept scores are sufficient statistics of the model prediction. Next, we propose a concept discovery method that aims to infer a complete set of concepts that are additionally encouraged to be interpretable, which addresses the limitations of existing methods on concept explanations. To define an importance score for each discovered concept, we adapt game-theoretic notions to aggregate over sets and propose ConceptSHAP. Via proposed metrics and user studies, on a synthetic dataset with apriori-known concept explanations, as well as on real-world image and language datasets, we validate the effectiveness of our method in finding concepts that are both complete in explaining the decisions and interpretable. (The code is released at https://github.com/chihkuanyeh/concept_exp)

arXiv commentUpdated supplementary

References in corpus (9)

• Understanding Black-box Predictions via Influence Functions
• Supervised Topic Models
• Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations
• SmoothGrad: removing noise by adding noise
• Learning to Generate Reviews and Discovering Sentiment
• Interpreting Black Box Predictions using Fisher Kernels
• Evaluating Explanation Without Ground Truth in Interpretable Machine Learning
• ProtoAttend: Attention-Based Prototypical Learning
• EDUCE: Explaining model Decisions through Unsupervised Concepts Extraction

Cited by in corpus (8)

• Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Survey
• Concept Whitening for Interpretable Image Recognition
• From Attribution Maps to Human-Understandable Explanations through Concept Relevance Propagation
• Now You See Me (CME): Concept-based Model Extraction
• SoK: Modeling Explainability in Security Analytics for Interpretability, Trustworthiness, and Usability
• Explaining Deep Neural Networks using Unsupervised Clustering
• Adversarial TCAV -- Robust and Effective Interpretation of Intermediate Layers in Neural Networks
• Corpus-level and Concept-based Explanations for Interpretable Document Classification