Towards a Comprehensive Human-Centred Evaluation Framework for Explainable AI

Abstract

While research on explainable AI (XAI) is booming and explanation techniques have proven promising in many application domains, standardised human-centred evaluation procedures are still missing. In addition, current evaluation procedures do not assess XAI methods holistically in the sense that they do not treat explanations' effects on humans as a complex user experience. To tackle this challenge, we propose to adapt the User-Centric Evaluation Framework used in recommender systems: we integrate explanation aspects, summarise explanation properties, indicate relations between them, and categorise metrics that measure these properties. With this comprehensive evaluation framework, we hope to contribute to the human-centred standardisation of XAI evaluation.

This preprint has not undergone any post-submission improvements or corrections. This work was an accepted contribution at the XAI world Conference 2023

References in corpus (4)

• What Do We Want From Explainable Artificial Intelligence (XAI)? -- A Stakeholder Perspective on XAI and a Conceptual Model Guiding Interdisciplinary XAI Research
• Proxy Tasks and Subjective Measures Can Be Misleading in Evaluating Explainable AI Systems
• Explainability Fact Sheets: A Framework for Systematic Assessment of Explainable Approaches
• Beyond Expertise and Roles: A Framework to Characterize the Stakeholders of Interpretable Machine Learning and their Needs

Cited by in corpus (1)

• What Does Evaluation of Explainable Artificial Intelligence Actually Tell Us? A Case for Compositional and Contextual Validation of XAI Building Blocks