'It's Reducing a Human Being to a Percentage'; Perceptions of Justice in Algorithmic Decisions
arXiv:1801.10408 · doi:10.1145/3173574.3173951
Abstract
Data-driven decision-making consequential to individuals raises important questions of accountability and justice. Indeed, European law provides individuals limited rights to 'meaningful information about the logic' behind significant, autonomous decisions such as loan approvals, insurance quotes, and CV filtering. We undertake three experimental studies examining people's perceptions of justice in algorithmic decision-making under different scenarios and explanation styles. Dimensions of justice previously observed in response to human decision-making appear similarly engaged in response to algorithmic decisions. Qualitative analysis identified several concerns and heuristics involved in justice perceptions including arbitrariness, generalisation, and (in)dignity. Quantitative analysis indicates that explanation styles primarily matter to justice perceptions only when subjects are exposed to multiple different styles---under repeated exposure of one style, scenario effects obscure any explanation effects. Our results suggests there may be no 'best' approach to explaining algorithmic decisions, and that reflection on their automated nature both implicates and mitigates justice dimensions.
14 pages, 3 figures, ACM Conference on Human Factors in Computing Systems (CHI'18), April 21--26, Montreal, Canada
References in corpus (1)
Cited by in corpus (6)
- Improving fairness in machine learning systems: What do industry practitioners need?
- Explaining Models: An Empirical Study of How Explanations Impact Fairness Judgment
- Trust in Data Science: Collaboration, Translation, and Accountability in Corporate Data Science Projects
- A Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scores
- This Thing Called Fairness: Disciplinary Confusion Realizing a Value in Technology
- Minimalistic Explanations: Capturing the Essence of Decisions