Representer Point Selection for Explaining Deep Neural Networks

Abstract

We propose to explain the predictions of a deep neural network, by pointing to the set of what we call representer points in the training set, for a given test point prediction. Specifically, we show that we can decompose the pre-activation prediction of a neural network into a linear combination of activations of training points, with the weights corresponding to what we call representer values, which thus capture the importance of that training point on the learned parameters of the network. But it provides a deeper understanding of the network than simply training point influence: with positive representer values corresponding to excitatory training points, and negative values corresponding to inhibitory points, which as we show provides considerably more insight. Our method is also much more scalable, allowing for real-time feedback in a manner not feasible with influence functions.

NIPS 2018

Cited by in corpus (35)

• A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAI
• GNNExplainer: Generating Explanations for Graph Neural Networks
• What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation
• On the (In)fidelity and Sensitivity for Explanations
• Machine Learning Explainability for External Stakeholders
• A Survey on Neural Network Interpretability
• HYDRA: Hypergradient Data Relevance Analysis for Interpreting Deep Neural Networks
• The Bouncer Problem: Challenges to Remote Explainability
• Towards Frequency-Based Explanation for Robust CNN
• Random Matrix Theory Proves that Deep Learning Representations of GAN-data Behave as Gaussian Mixtures
• Certified Robustness to Label-Flipping Attacks via Randomized Smoothing
• Evaluation of Similarity-based Explanations
• Evaluations and Methods for Explanation through Robustness Analysis
• Interpretable Machine Learning: Moving From Mythos to Diagnostics
• Understanding Instance-based Interpretability of Variational Auto-Encoders
• Sparse Oblique Decision Trees: A Tool to Understand and Manipulate Neural Net Features
• Attribute Prototype Network for Zero-Shot Learning
• Debugging Tests for Model Explanations
• Data Representing Ground-Truth Explanations to Evaluate XAI Methods
• A Survey on Understanding, Visualizations, and Explanation of Deep Neural Networks
• A Unified Framework for Task-Driven Data Quality Management
• DIVINE: Diverse Influential Training Points for Data Visualization and Model Refinement
• Interactive Label Cleaning with Example-based Explanations
• Interpreting Deep Models through the Lens of Data
• Sampling the "Inverse Set" of a Neuron: An Approach to Understanding Neural Nets
• On Sample Based Explanation Methods for NLP:Efficiency, Faithfulness, and Semantic Evaluation
• TREX: Tree-Ensemble Representer-Point Explanations
• EDDA: Explanation-driven Data Augmentation to Improve Explanation Faithfulness
• Explaining the Road Not Taken
• DANCE: Enhancing saliency maps using decoys
• Model-Agnostic Explanations using Minimal Forcing Subsets
• Longitudinal Distance: Towards Accountable Instance Attribution
• Influence Tuning: Demoting Spurious Correlations via Instance Attribution and Instance-Driven Updates
• Automated Dependence Plots
• Explaining Predictions of Deep Neural Classifier via Activation Analysis