Best of both worlds: local and global explanations with human-understandable concepts
arXiv:2106.08641
Abstract
Interpretability techniques aim to provide the rationale behind a model's decision, typically by explaining either an individual prediction (local explanation, e.g. 'why is this patient diagnosed with this condition') or a class of predictions (global explanation, e.g. 'why is this set of patients diagnosed with this condition in general'). While there are many methods focused on either one, few frameworks can provide both local and global explanations in a consistent manner. In this work, we combine two powerful existing techniques, one local (Integrated Gradients, IG) and one global (Testing with Concept Activation Vectors), to provide local and global concept-based explanations. We first sanity check our idea using two synthetic datasets with a known ground truth, and further demonstrate with a benchmark natural image dataset. We test our method with various concepts, target classes, model architectures and IG parameters (e.g. baselines). We show that our method improves global explanations over vanilla TCAV when compared to ground truth, and provides useful local insights. Finally, a user study demonstrates the usefulness of the method compared to no or global explanations only. We hope our work provides a step towards building bridges between many existing local and global methods to get the best of both worlds.
References in corpus (15)
- Adam: A Method for Stochastic Optimization
- Towards A Rigorous Science of Interpretable Machine Learning
- Axiomatic Attribution for Deep Networks
- Learning Important Features Through Propagating Activation Differences
- SmoothGrad: removing noise by adding noise
- Sanity Checks for Saliency Maps
- Not Just a Black Box: Learning Important Features Through Propagating Activation Differences
- Explainable AI for Trees: From Local Explanations to Global Understanding
- What Clinicians Want: Contextualizing Explainable Machine Learning for Clinical End Use
- The (Un)reliability of saliency methods
- Concept Bottleneck Models
- Explanations can be manipulated and geometry is to blame
- Explaining Classifiers with Causal Concept Effect (CaCE)
- Understanding Integrated Gradients with SmoothTaylor for Deep Neural Network Attribution
- Global Aggregations of Local Explanations for Black Box models