EXMOS: Explanatory Model Steering Through Multifaceted Explanations and Data Configurations

Abstract

Explanations in interactive machine-learning systems facilitate debugging and improving prediction models. However, the effectiveness of various global model-centric and data-centric explanations in aiding domain experts to detect and resolve potential data issues for model improvement remains unexplored. This research investigates the influence of data-centric and model-centric global explanations in systems that support healthcare experts in optimising models through automated and manual data configurations. We conducted quantitative (n=70) and qualitative (n=30) studies with healthcare experts to explore the impact of different explanations on trust, understandability and model improvement. Our results reveal the insufficiency of global model-centric explanations for guiding users during data configuration. Although data-centric explanations enhanced understanding of post-configuration system changes, a hybrid fusion of both explanation types demonstrated the highest effectiveness. Based on our study results, we also present design implications for effective explanation-driven interactive machine-learning systems.

arXiv commentThis is a pre-print version only for early release. Please view the conference published version from ACM CHI 2024 to get the latest version of the paper

References in corpus (11)

• Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization
• Questioning the AI: Informing Design Practices for Explainable AI User Experiences
• explAIner: A Visual Analytics Framework for Interactive and Explainable Machine Learning
• Explaining Models: An Empirical Study of How Explanations Impact Fairness Judgment
• "Help Me Help the AI": Understanding How Explainability Can Support Human-AI Interaction
• Interpretable Classification Models for Recidivism Prediction
• Directive Explanations for Monitoring the Risk of Diabetes Onset: Introducing Directive Data-Centric Explanations and Combinations to Support What-If Explorations
• Interpretability, Then What? Editing Machine Learning Models to Reflect Human Knowledge and Values
• DeepSeer: Interactive RNN Explanation and Debugging via State Abstraction
• A Nested Genetic Algorithm for Explaining Classification Data Sets with Decision Rules
• Explainers in the Wild: Making Surrogate Explainers Robust to Distortions through Perception

Cited by in corpus (7)

• An Explanatory Model Steering System for Collaboration between Domain Experts and AI
• Representation Debiasing of Generated Data Involving Domain Experts
• Show Me How: Benefits and Challenges of Agent-Augmented Counterfactual Explanations for Non-Expert Users
• Explanatory Debiasing: Involving Domain Experts in the Data Generation Process to Mitigate Representation Bias in AI Systems
• Abstraction Alignment: Comparing Model-Learned and Human-Encoded Conceptual Relationships
• Leveraging Complementary AI Explanations to Mitigate Misunderstanding in XAI
• Tempo: Helping Data Scientists and Domain Experts Collaboratively Specify Predictive Modeling Tasks