Quantifying Interpretability and Trust in Machine Learning Systems

Abstract

Decisions by Machine Learning (ML) models have become ubiquitous. Trusting these decisions requires understanding how algorithms take them. Hence interpretability methods for ML are an active focus of research. A central problem in this context is that both the quality of interpretability methods as well as trust in ML predictions are difficult to measure. Yet evaluations, comparisons and improvements of trust and interpretability require quantifiable measures. Here we propose a quantitative measure for the quality of interpretability methods. Based on that we derive a quantitative measure of trust in ML decisions. Building on previous work we propose to measure intuitive understanding of algorithmic decisions using the information transfer rate at which humans replicate ML model predictions. We provide empirical evidence from crowdsourcing experiments that the proposed metric robustly differentiates interpretability methods. The proposed metric also demonstrates the value of interpretability for ML assisted human decision making: in our experiments providing explanations more than doubled productivity in annotation tasks. However unbiased human judgement is critical for doctors, judges, policy makers and others. Here we derive a trust metric that identifies when human decisions are overly biased towards ML predictions. Our results complement existing qualitative work on trust and interpretability by quantifiable measures that can serve as objectives for further improving methods in this field of research.

In AAAI-19 Workshop on Network Interpretability for Deep Learning

References in corpus (1)

• The Promise and Peril of Human Evaluation for Model Interpretability

Cited by in corpus (17)

• A Survey on the Explainability of Supervised Machine Learning
• Human Factors in Model Interpretability: Industry Practices, Challenges, and Needs
• How can I choose an explainer? An Application-grounded Evaluation of Post-hoc Explanations
• Counterfactual Explanations for Machine Learning on Multivariate Time Series Data
• Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AI
• A Human-Grounded Evaluation Benchmark for Local Explanations of Machine Learning
• The effectiveness of feature attribution methods and its correlation with automatic evaluation scores
• Micro-entries: Encouraging Deeper Evaluation of Mental Models Over Time for Interactive Data Systems
• RecRec: Algorithmic Recourse for Recommender Systems
• Exemplary Natural Images Explain CNN Activations Better than State-of-the-Art Feature Visualization
• A psychophysics approach for quantitative comparison of interpretable computer vision models
• Technologies for Trustworthy Machine Learning: A Survey in a Socio-Technical Context
• Quality Metrics for Transparent Machine Learning With and Without Humans In the Loop Are Not Correlated
• Intelligent Decision Assistance Versus Automated Decision-Making: Enhancing Knowledge Work Through Explainable Artificial Intelligence
• Human-Imitating Metrics for Training and Evaluating Privacy Preserving Emotion Recognition Models Using Sociolinguistic Knowledge
• Spoofing Generalization: When Can't You Trust Proprietary Models?
• A Turing Test for Transparency