What can AI do for me: Evaluating Machine Learning Interpretations in Cooperative Play

Abstract

Machine learning is an important tool for decision making, but its ethical and responsible application requires rigorous vetting of its interpretability and utility: an understudied problem, particularly for natural language processing models. We propose an evaluation of interpretation on a real task with real human users, where the effectiveness of interpretation is measured by how much it improves human performance. We design a grounded, realistic human-computer cooperative setting using a question answering task, Quizbowl. We recruit both trivia experts and novices to play this game with computer as their teammate, who communicates its prediction via three different interpretations. We also provide design guidance for natural language processing human-in-the-loop settings.

References in corpus (18)

• Towards A Rigorous Science of Interpretable Machine Learning
• Axiomatic Attribution for Deep Networks
• On Calibration of Modern Neural Networks
• A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
• Understanding Black-box Predictions via Influence Functions
• Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model
• How to Explain Individual Classification Decisions
• Sanity Checks for Saliency Maps
• Understanding Neural Networks through Representation Erasure
• Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning
• On Human Predictions with Explanations and Predictions of Machine Learning Models: A Case Study on Deception Detection
• Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradients
• To Trust Or Not To Trust A Classifier
• Pathologies of Neural Models Make Interpretations Difficult
• Beyond Word Importance: Contextual Decomposition to Extract Interactions from LSTMs
• The (Un)reliability of saliency methods
• How do Humans Understand Explanations from Machine Learning Systems? An Evaluation of the Human-Interpretability of Explanation
• AI Challenges in Human-Robot Cognitive Teaming

Cited by in corpus (6)

• On Human Predictions with Explanations and Predictions of Machine Learning Models: A Case Study on Deception Detection
• Towards Human-centered Explainable AI: A Survey of User Studies for Model Explanations
• How can I choose an explainer? An Application-grounded Evaluation of Post-hoc Explanations
• Human-AI Collaboration: The Effect of AI Delegation on Human Task Performance and Task Satisfaction
• Understanding the Effect of Counterfactual Explanations on Trust and Reliance on AI for Human-AI Collaborative Clinical Decision Making
• Evaluating Explanation Without Ground Truth in Interpretable Machine Learning