'It's Reducing a Human Being to a Percentage'; Perceptions of Justice in Algorithmic Decisions

Abstract

Data-driven decision-making consequential to individuals raises important questions of accountability and justice. Indeed, European law provides individuals limited rights to 'meaningful information about the logic' behind significant, autonomous decisions such as loan approvals, insurance quotes, and CV filtering. We undertake three experimental studies examining people's perceptions of justice in algorithmic decision-making under different scenarios and explanation styles. Dimensions of justice previously observed in response to human decision-making appear similarly engaged in response to algorithmic decisions. Qualitative analysis identified several concerns and heuristics involved in justice perceptions including arbitrariness, generalisation, and (in)dignity. Quantitative analysis indicates that explanation styles primarily matter to justice perceptions only when subjects are exposed to multiple different styles---under repeated exposure of one style, scenario effects obscure any explanation effects. Our results suggests there may be no 'best' approach to explaining algorithmic decisions, and that reflection on their automated nature both implicates and mitigates justice dimensions.

14 pages, 3 figures, ACM Conference on Human Factors in Computing Systems (CHI'18), April 21--26, Montreal, Canada

References in corpus (1)

• Towards A Rigorous Science of Interpretable Machine Learning

Cited by in corpus (51)

• Improving fairness in machine learning systems: What do industry practitioners need?
• What Do We Want From Explainable Artificial Intelligence (XAI)? -- A Stakeholder Perspective on XAI and a Conceptual Model Guiding Interdisciplinary XAI Research
• Explanations in Autonomous Driving: A Survey
• Bias and Discrimination in AI: a cross-disciplinary perspective
• Explaining Models: An Empirical Study of How Explanations Impact Fairness Judgment
• Towards Human-centered Explainable AI: A Survey of User Studies for Model Explanations
• Explainable Artificial Intelligence (XAI) from a user perspective- A synthesis of prior literature and problematizing avenues for future research
• Trust in Data Science: Collaboration, Translation, and Accountability in Corporate Data Science Projects
• A Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scores
• This Thing Called Fairness: Disciplinary Confusion Realizing a Value in Technology
• Investigating How Practitioners Use Human-AI Guidelines: A Case Study on the People + AI Guidebook
• Is Stack Overflow Obsolete? An Empirical Study of the Characteristics of ChatGPT Answers to Stack Overflow Questions
• The Who in XAI: How AI Background Shapes Perceptions of AI Explanations
• XAIR: A Framework of Explainable AI in Augmented Reality
• Explanations, Fairness, and Appropriate Reliance in Human-AI Decision-Making
• Comparing the Perceived Legitimacy of Content Moderation Processes: Contractors, Algorithms, Expert Panels, and Digital Juries
• "There Is Not Enough Information": On the Effects of Explanations on Perceptions of Informational Fairness and Trustworthiness in Automated Decision-Making
• How fair can we go in machine learning? Assessing the boundaries of fairness in decision trees
• Soliciting Stakeholders' Fairness Notions in Child Maltreatment Predictive Systems
• "Look! It's a Computer Program! It's an Algorithm! It's AI!": Does Terminology Affect Human Perceptions and Evaluations of Algorithmic Decision-Making Systems?
• Dissonance Between Human and Machine Understanding
• Certification Labels for Trustworthy AI: Insights From an Empirical Mixed-Method Study
• A Critical Survey on Fairness Benefits of Explainable AI
• How Platform-User Power Relations Shape Algorithmic Accountability: A Case Study of Instant Loan Platforms and Financially Stressed Users in India
• The Algorithmic Imprint
• Inherent Limitations of AI Fairness
• A Framework and Benchmarking Study for Counterfactual Generating Methods on Tabular Data
• Responding to Generative AI Technologies with Research-through-Design: The Ryelands AI Lab as an Exploratory Study
• Out of Context: Investigating the Bias and Fairness Concerns of "Artificial Intelligence as a Service"
• Participation and Division of Labor in User-Driven Algorithm Audits: How Do Everyday Users Work together to Surface Algorithmic Harms?
• Appropriate Fairness Perceptions? On the Effectiveness of Explanations in Enabling People to Assess the Fairness of Automated Decision Systems
• Explainability Auditing for Intelligent Systems: A Rationale for Multi-Disciplinary Perspectives
• Co-Writing with AI, on Human Terms: Aligning Research with User Demands Across the Writing Process
• Explanation Strategies as an Empirical-Analytical Lens for Socio-Technical Contextualization of Machine Learning Interpretability
• The Impact of Explanations on Fairness in Human-AI Decision-Making: Protected vs Proxy Features
• A Critical Reflection on the Use of Toxicity Detection Algorithms in Proactive Content Moderation Systems
• Towards the Socio-Algorithmic Construction of Fairness: The Case of Automatic Price-Surging in Ride-Hailing
• Comprehension Is a Double-Edged Sword: Over-Interpreting Unspecified Information in Intelligible Machine Learning Explanations
• Access Denied: Meaningful Data Access for Quantitative Algorithm Audits
• A Human-Centric Perspective on Fairness and Transparency in Algorithmic Decision-Making
• Navigating Automated Hiring: Perceptions, Strategy Use, and Outcomes Among Young Job Seekers
• Minimalistic Explanations: Capturing the Essence of Decisions
• It's only fair when I think it's fair: How Gender Bias Alignment Undermines Distributive Fairness in Human-AI Collaboration
• A Matter of Perspective(s): Contrasting Human and LLM Argumentation in Subjective Decision-Making on Subtle Sexism
• Time Can Invalidate Algorithmic Recourse
• (Ir)rationality in AI: State of the Art, Research Challenges and Open Questions
• Towards a multi-stakeholder value-based assessment framework for algorithmic systems
• A Human-Centered Privacy Approach (HCP) to AI
• Explainability's Gain is Optimality's Loss? -- How Explanations Bias Decision-making
• Regulating Social Media: Surveying the Impact of Nepali Government's TikTok Ban
• Outlining Traceability: A Principle for Operationalizing Accountability in Computing Systems