Interpreting Black Box Predictions using Fisher Kernels

Abstract

Research in both machine learning and psychology suggests that salient examples can help humans to interpret learning models. To this end, we take a novel look at black box interpretation of test predictions in terms of training examples. Our goal is to ask `which training examples are most responsible for a given set of predictions'? To answer this question, we make use of Fisher kernels as the defining feature embedding of each data point, combined with Sequential Bayesian Quadrature (SBQ) for efficient selection of examples. In contrast to prior work, our method is able to seamlessly handle any sized subset of test predictions in a principled way. We theoretically analyze our approach, providing novel convergence bounds for SBQ over discrete candidate atoms. Our approach recovers the application of influence functions for interpretability as a special case yielding novel insights from this connection. We also present applications of the proposed approach to three use cases: cleaning training data, fixing mislabeled examples and data summarization.

Cited by in corpus (18)

• Towards Explainable Artificial Intelligence
• Explainable Artificial Intelligence: a Systematic Review
• On the Accuracy of Influence Functions for Measuring Group Effects
• On the (In)fidelity and Sensitivity for Explanations
• On Completeness-aware Concept-Based Explanations in Deep Neural Networks
• Machine Learning Explainability for External Stakeholders
• Data Cleansing for Models Trained with SGD
• Evaluation of Similarity-based Explanations
• Generative causal explanations of black-box classifiers
• Understanding Instance-based Interpretability of Variational Auto-Encoders
• Multi-Stage Influence Function
• Influence Estimation for Generative Adversarial Networks
• Knowledge-based XAI through CBR: There is more to explanations than models can tell
• DIVINE: Diverse Influential Training Points for Data Visualization and Model Refinement
• Interactive Label Cleaning with Example-based Explanations
• Fortifying Toxic Speech Detectors Against Veiled Toxicity
• Unifying Model Explainability and Robustness via Machine-Checkable Concepts
• Longitudinal Distance: Towards Accountable Instance Attribution