Evaluation of Similarity-based Explanations

Abstract

Explaining the predictions made by complex machine learning models helps users to understand and accept the predicted outputs with confidence. One promising way is to use similarity-based explanation that provides similar instances as evidence to support model predictions. Several relevance metrics are used for this purpose. In this study, we investigated relevance metrics that can provide reasonable explanations to users. Specifically, we adopted three tests to evaluate whether the relevance metrics satisfy the minimal requirements for similarity-based explanation. Our experiments revealed that the cosine similarity of the gradients of the loss performs best, which would be a recommended choice in practice. In addition, we showed that some metrics perform poorly in our tests and analyzed the reasons of their failure. We expect our insights to help practitioners in selecting appropriate relevance metrics and also aid further researches for designing better relevance metrics for explanations.

arXiv commentICLR 2021

References in corpus (11)

• Distributed Representations of Words and Phrases and their Compositionality
• UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction
• Towards A Rigorous Science of Interpretable Machine Learning
• Understanding Black-box Predictions via Influence Functions
• Distance and Similarity Measures Effect on the Performance of K-Nearest Neighbor Classifier -- A Review
• Sanity Checks for Saliency Maps
• Prototype selection for interpretable classification
• An Evaluation of the Human-Interpretability of Explanation
• Representer Point Selection for Explaining Deep Neural Networks
• Interpreting Black Box Predictions using Fisher Kernels
• Input Similarity from the Neural Network Perspective

Cited by in corpus (5)

• Understanding Instance-based Interpretability of Variational Auto-Encoders
• The Thousand Faces of Explainable AI Along the Machine Learning Life Cycle: Industrial Reality and Current State of Research
• Advancing Post Hoc Case Based Explanation with Feature Highlighting
• An Empirical Comparison of Instance Attribution Methods for NLP
• Adversarial Attacks on Knowledge Graph Embeddings via Instance Attribution Methods