Beyond Expertise and Roles: A Framework to Characterize the Stakeholders of Interpretable Machine Learning and their Needs

Abstract

To ensure accountability and mitigate harm, it is critical that diverse stakeholders can interrogate black-box automated systems and find information that is understandable, relevant, and useful to them. In this paper, we eschew prior expertise- and role-based categorizations of interpretability stakeholders in favor of a more granular framework that decouples stakeholders' knowledge from their interpretability needs. We characterize stakeholders by their formal, instrumental, and personal knowledge and how it manifests in the contexts of machine learning, the data domain, and the general milieu. We additionally distill a hierarchical typology of stakeholder needs that distinguishes higher-level domain goals from lower-level interpretability tasks. In assessing the descriptive, evaluative, and generative powers of our framework, we find our more nuanced treatment of stakeholders reveals gaps and opportunities in the interpretability literature, adds precision to the design and comparison of user studies, and facilitates a more reflexive approach to conducting this research.

arXiv commentIn CHI Conference on Human Factors in Computing Systems (CHI '21)

References in corpus (16)

• Model Cards for Model Reporting
• Explainable Machine Learning for Scientific Insights and Discoveries
• Questioning the AI: Informing Design Practices for Explainable AI User Experiences
• Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making
• Deep Recurrent Quantization for Generating Sequential Binary Codes
• Beyond Product Quantization: Deep Progressive Quantization for Image Retrieval
• The What-If Tool: Interactive Probing of Machine Learning Models
• Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)
• Actionable Recourse in Linear Classification
• A systematic review and taxonomy of explanations in decision support and recommender systems
• On Human Predictions with Explanations and Predictions of Machine Learning Models: A Case Study on Deception Detection
• Explaining Models: An Empirical Study of How Explanations Impact Fairness Judgment
• Human Factors in Model Interpretability: Industry Practices, Challenges, and Needs
• Ethical Dimensions of Visualization Research
• Misplaced Trust: Measuring the Interference of Machine Learning in Human Decision-Making
• An Exploratory Study on Visual Exploration of Model Simulations by Multiple Types of Experts

Cited by in corpus (20)

• The Metacognitive Demands and Opportunities of Generative AI
• "Help Me Help the AI": Understanding How Explainability Can Support Human-AI Interaction
• Fairness via Explanation Quality: Evaluating Disparities in the Quality of Post hoc Explanations
• Telling Stories from Computational Notebooks: AI-Assisted Presentation Slides Creation for Presenting Data Science Work
• Machine Learning Security in Industry: A Quantitative Survey
• Seamful XAI: Operationalizing Seamful Design in Explainable AI
• Knowledge Graphs in Practice: Characterizing their Users, Challenges, and Visualization Opportunities
• Let's Talk Futures: A Literature Review of HCI's Future-Orientation
• GAM Coach: Towards Interactive and User-centered Algorithmic Recourse
• Can Requirements Engineering Support Explainable Artificial Intelligence? Towards a User-Centric Approach for Explainability Requirements
• Towards a Comprehensive Human-Centred Evaluation Framework for Explainable AI
• Saliency Cards: A Framework to Characterize and Compare Saliency Methods
• Secondary Stakeholders in AI: Fighting for, Brokering, and Navigating Agency
• Are Metrics Enough? Guidelines for Communicating and Visualizing Predictive Models to Subject Matter Experts
• (Beyond) Reasonable Doubt: Challenges that Public Defenders Face in Scrutinizing AI in Court
• Identifying Explanation Needs of End-users: Applying and Extending the XAI Question Bank
• Abstraction Alignment: Comparing Model-Learned and Human-Encoded Conceptual Relationships
• Towards Uncertainty Aware Task Delegation and Human-AI Collaborative Decision-Making
• Can AI Model the Complexities of Human Moral Decision-Making? A Qualitative Study of Kidney Allocation Decisions
• Towards a multi-stakeholder value-based assessment framework for algorithmic systems