Factors Influencing Perceived Fairness in Algorithmic Decision-Making: Algorithm Outcomes, Development Procedures, and Individual Differences

Abstract

Algorithmic decision-making systems are increasingly used throughout the public and private sectors to make important decisions or assist humans in making these decisions with real social consequences. While there has been substantial research in recent years to build fair decision-making algorithms, there has been less research seeking to understand the factors that affect people's perceptions of fairness in these systems, which we argue is also important for their broader acceptance. In this research, we conduct an online experiment to better understand perceptions of fairness, focusing on three sets of factors: algorithm outcomes, algorithm development and deployment procedures, and individual differences. We find that people rate the algorithm as more fair when the algorithm predicts in their favor, even surpassing the negative effects of describing algorithms that are very biased against particular demographic groups. We find that this effect is moderated by several variables, including participants' education level, gender, and several aspects of the development procedure. Our findings suggest that systems that evaluate algorithmic fairness through users' feedback must consider the possibility of outcome favorability bias.

To appear at CHI 2020

References in corpus (6)

• Understanding Black-box Predictions via Influence Functions
• Fairness and Accountability Design Needs for Algorithmic Support in High-Stakes Public Sector Decision-Making
• The Frontiers of Fairness in Machine Learning
• A Reductions Approach to Fair Classification
• Demographics and discussion influence views on algorithmic fairness
• Mathematical Notions vs. Human Perception of Fairness: A Descriptive Approach to Fairness for Machine Learning

Cited by in corpus (5)

• Exploring How Machine Learning Practitioners (Try To) Use Fairness Toolkits
• Human Perceptions on Moral Responsibility of AI: A Case Study in AI-Assisted Bail Decision-Making
• Competent but Rigid: Identifying the Gap in Empowering AI to Participate Equally in Group Decision-Making
• Inherent Limitations of AI Fairness
• Appropriate Fairness Perceptions? On the Effectiveness of Explanations in Enabling People to Assess the Fairness of Automated Decision Systems