Explainable Active Learning (XAL): An Empirical Study of How Local Explanations Impact Annotator Experience

Abstract

The wide adoption of Machine Learning technologies has created a rapidly growing demand for people who can train ML models. Some advocated the term "machine teacher" to refer to the role of people who inject domain knowledge into ML models. One promising learning paradigm is Active Learning (AL), by which the model intelligently selects instances to query the machine teacher for labels. However, in current AL settings, the human-AI interface remains minimal and opaque. We begin considering AI explanations as a core element of the human-AI interface for teaching machines. When a human student learns, it is a common pattern to present one's own reasoning and solicit feedback from the teacher. When a ML model learns and still makes mistakes, the human teacher should be able to understand the reasoning underlying the mistakes. When the model matures, the machine teacher should be able to recognize its progress in order to trust and feel confident about their teaching outcome. Toward this vision, we propose a novel paradigm of explainable active learning (XAL), by introducing techniques from the recently surging field of explainable AI (XAI) into an AL setting. We conducted an empirical study comparing the model learning outcomes, feedback content and experience with XAL, to that of traditional AL and coactive learning (providing the model's prediction without the explanation). Our study shows benefits of AI explanation as interfaces for machine teaching--supporting trust calibration and enabling rich forms of teaching feedback, and potential drawbacks--anchoring effect with the model judgment and cognitive workload. Our study also reveals important individual factors that mediate a machine teacher's reception to AI explanations, including task knowledge, AI experience and need for cognition. By reflecting on the results, we suggest future directions and design implications for XAL.

arXiv commentreplacing with a draft accepted to CSCW2020

References in corpus (14)

• Towards A Rigorous Science of Interpretable Machine Learning
• The What-If Tool: Interactive Probing of Machine Learning Models
• On Human Predictions with Explanations and Predictions of Machine Learning Models: A Case Study on Deception Detection
• One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniques
• Prototype selection for interpretable classification
• Manifold: A Model-Agnostic Framework for Interpretation and Diagnosis of Machine Learning Models
• A Benchmark and Comparison of Active Learning for Logistic Regression
• How do Humans Understand Explanations from Machine Learning Systems? An Evaluation of the Human-Interpretability of Explanation
• Interpretable to Whom? A Role-based Model for Analyzing Interpretable Machine Learning Systems
• Machine Teaching: A New Paradigm for Building Machine Learning Systems
• Stopping Active Learning based on Predicted Change of F Measure for Text Classification
• The Emerging Landscape of Explainable AI Planning and Decision Making
• Impact of Batch Size on Stopping Active Learning for Text Classification
• "Why Should I Trust Interactive Learners?" Explaining Interactive Queries of Classifiers to Users

Cited by in corpus (7)

• Facilitating Knowledge Sharing from Domain Experts to Data Scientists for Building NLP Models
• A Survey on Cost Types, Interaction Schemes, and Annotator Performance Models in Selection Algorithms for Active Learning in Classification
• Human-Centered Explainable AI (XAI): From Algorithms to User Experiences
• Show or Suppress? Managing Input Uncertainty in Machine Learning Model Explanations
• Class Introspection: A Novel Technique for Detecting Unlabeled Subclasses by Leveraging Classifier Explainability Methods
• ALEX: Active Learning based Enhancement of a Model's Explainability
• Active Learning++: Incorporating Annotator's Rationale using Local Model Explanation