Explaining Models: An Empirical Study of How Explanations Impact Fairness Judgment

Abstract

Ensuring fairness of machine learning systems is a human-in-the-loop process. It relies on developers, users, and the general public to identify fairness problems and make improvements. To facilitate the process we need effective, unbiased, and user-friendly explanations that people can confidently rely on. Towards that end, we conducted an empirical study with four types of programmatically generated explanations to understand how they impact people's fairness judgments of ML systems. With an experiment involving more than 160 Mechanical Turk workers, we show that: 1) Certain explanations are considered inherently less fair, while others can enhance people's confidence in the fairness of the algorithm; 2) Different fairness problems--such as model-wide fairness issues versus case-specific fairness discrepancies--may be more effectively exposed through different styles of explanation; 3) Individual differences, including prior positions and judgment criteria of algorithmic fairness, impact how people react to different styles of explanation. We conclude with a discussion on providing personalized and adaptive explanations to support fairness judgments of ML systems.

References in corpus (4)

• 'It's Reducing a Human Being to a Percentage'; Perceptions of Justice in Algorithmic Decisions
• Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatment
• Fairness and Accountability Design Needs for Algorithmic Support in High-Stakes Public Sector Decision-Making
• Fairness in Learning: Classic and Contextual Bandits

Cited by in corpus (30)

• Questioning the AI: Informing Design Practices for Explainable AI User Experiences
• Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making
• Expanding Explainability: Towards Social Transparency in AI systems
• Towards Human-centered Explainable AI: A Survey of User Studies for Model Explanations
• Explainable Artificial Intelligence (XAI) from a user perspective- A synthesis of prior literature and problematizing avenues for future research
• Designing AI for Trust and Collaboration in Time-Constrained Medical Decisions: A Sociotechnical Lens
• Beyond Expertise and Roles: A Framework to Characterize the Stakeholders of Interpretable Machine Learning and their Needs
• The Who in XAI: How AI Background Shapes Perceptions of AI Explanations
• Charting the Sociotechnical Gap in Explainable AI: A Framework to Address the Gap in XAI
• Explanations, Fairness, and Appropriate Reliance in Human-AI Decision-Making
• Fairness and Decision-making in Collaborative Shift Scheduling Systems
• "There Is Not Enough Information": On the Effects of Explanations on Perceptions of Informational Fairness and Trustworthiness in Automated Decision-Making
• Fairness via Explanation Quality: Evaluating Disparities in the Quality of Post hoc Explanations
• Soliciting Stakeholders' Fairness Notions in Child Maltreatment Predictive Systems
• A Critical Survey on Fairness Benefits of Explainable AI
• Seamful XAI: Operationalizing Seamful Design in Explainable AI
• EXMOS: Explanatory Model Steering Through Multifaceted Explanations and Data Configurations
• Humans, AI, and Context: Understanding End-Users' Trust in a Real-World Computer Vision Application
• A Framework and Benchmarking Study for Counterfactual Generating Methods on Tabular Data
• FATE in AI: Towards Algorithmic Inclusivity and Accessibility
• Appropriate Fairness Perceptions? On the Effectiveness of Explanations in Enabling People to Assess the Fairness of Automated Decision Systems
• Explanation Strategies as an Empirical-Analytical Lens for Socio-Technical Contextualization of Machine Learning Interpretability
• Show Me the Work: Fact-Checkers' Requirements for Explainable Automated Fact-Checking
• The Impact of Explanations on Fairness in Human-AI Decision-Making: Protected vs Proxy Features
• A Human-Centric Perspective on Fairness and Transparency in Algorithmic Decision-Making
• DeepSeer: Interactive RNN Explanation and Debugging via State Abstraction
• Dissecting users' needs for search result explanations
• It's only fair when I think it's fair: How Gender Bias Alignment Undermines Distributive Fairness in Human-AI Collaboration
• "Even explanations will not help in trusting [this] fundamentally biased system": A Predictive Policing Case-Study
• Towards a multi-stakeholder value-based assessment framework for algorithmic systems