On quantitative aspects of model interpretability

Abstract

Despite the growing body of work in interpretable machine learning, it remains unclear how to evaluate different explainability methods without resorting to qualitative assessment and user-studies. While interpretability is an inherently subjective matter, previous works in cognitive science and epistemology have shown that good explanations do possess aspects that can be objectively judged apart from fidelity), such assimplicity and broadness. In this paper we propose a set of metrics to programmatically evaluate interpretability methods along these dimensions. In particular, we argue that the performance of methods along these dimensions can be orthogonally imputed to two conceptual parts, namely the feature extractor and the actual explainability method. We experimentally validate our metrics on different benchmark tasks and show how they can be used to guide a practitioner in the selection of the most appropriate method for the task at hand.

References in corpus (8)

• Estimating Mutual Information
• Towards A Rigorous Science of Interpretable Machine Learning
• Axiomatic Attribution for Deep Networks
• A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAI
• Towards Robust Interpretability with Self-Explaining Neural Networks
• A Benchmark for Interpretability Methods in Deep Neural Networks
• Investigating the influence of noise and distractors on the interpretation of neural networks
• bLIMEy: Surrogate Prediction Explanations Beyond LIME

Cited by in corpus (6)

• Synthetic Benchmarks for Scientific Research in Explainable Machine Learning
• Pitfalls of Explainable ML: An Industry Perspective
• Achievements and Challenges in Explaining Deep Learning based Computer-Aided Diagnosis Systems
• GANMEX: One-vs-One Attributions Guided by GAN-based Counterfactual Explanation Baselines
• Explaining Deep Learning Representations by Tracing the Training Process
• Thermostat: A Large Collection of NLP Model Explanations and Analysis Tools