Explaining Models: An Empirical Study of How Explanations Impact Fairness Judgment

Abstract

Ensuring fairness of machine learning systems is a human-in-the-loop process. It relies on developers, users, and the general public to identify fairness problems and make improvements. To facilitate the process we need effective, unbiased, and user-friendly explanations that people can confidently rely on. Towards that end, we conducted an empirical study with four types of programmatically generated explanations to understand how they impact people's fairness judgments of ML systems. With an experiment involving more than 160 Mechanical Turk workers, we show that: 1) Certain explanations are considered inherently less fair, while others can enhance people's confidence in the fairness of the algorithm; 2) Different fairness problems--such as model-wide fairness issues versus case-specific fairness discrepancies--may be more effectively exposed through different styles of explanation; 3) Individual differences, including prior positions and judgment criteria of algorithmic fairness, impact how people react to different styles of explanation. We conclude with a discussion on providing personalized and adaptive explanations to support fairness judgments of ML systems.

References in corpus (2)

• 'It's Reducing a Human Being to a Percentage'; Perceptions of Justice in Algorithmic Decisions
• Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatment

Cited by in corpus (16)

• Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making
• Expanding Explainability: Towards Social Transparency in AI systems
• Explainable Artificial Intelligence (XAI) from a user perspective- A synthesis of prior literature and problematizing avenues for future research
• Designing AI for Trust and Collaboration in Time-Constrained Medical Decisions: A Sociotechnical Lens
• Beyond Expertise and Roles: A Framework to Characterize the Stakeholders of Interpretable Machine Learning and their Needs
• Charting the Sociotechnical Gap in Explainable AI: A Framework to Address the Gap in XAI
• Fairness and Decision-making in Collaborative Shift Scheduling Systems
• "There Is Not Enough Information": On the Effects of Explanations on Perceptions of Informational Fairness and Trustworthiness in Automated Decision-Making
• Fairness via Explanation Quality: Evaluating Disparities in the Quality of Post hoc Explanations
• Soliciting Stakeholders' Fairness Notions in Child Maltreatment Predictive Systems
• EXMOS: Explanatory Model Steering Through Multifaceted Explanations and Data Configurations
• Humans, AI, and Context: Understanding End-Users' Trust in a Real-World Computer Vision Application
• Appropriate Fairness Perceptions? On the Effectiveness of Explanations in Enabling People to Assess the Fairness of Automated Decision Systems
• A Human-Centric Perspective on Fairness and Transparency in Algorithmic Decision-Making
• DeepSeer: Interactive RNN Explanation and Debugging via State Abstraction
• Dissecting users' needs for search result explanations