Equity of Attention: Amortizing Individual Fairness in Rankings

Abstract

Rankings of people and items are at the heart of selection-making, match-making, and recommender systems, ranging from employment sites to sharing economy platforms. As ranking positions influence the amount of attention the ranked subjects receive, biases in rankings can lead to unfair distribution of opportunities and resources, such as jobs or income. This paper proposes new measures and mechanisms to quantify and mitigate unfairness from a bias inherent to all rankings, namely, the position bias, which leads to disproportionately less attention being paid to low-ranked subjects. Our approach differs from recent fair ranking approaches in two important ways. First, existing works measure unfairness at the level of subject groups while our measures capture unfairness at the level of individual subjects, and as such subsume group unfairness. Second, as no single ranking can achieve individual attention fairness, we propose a novel mechanism that achieves amortized fairness, where attention accumulated across a series of rankings is proportional to accumulated relevance. We formulate the challenge of achieving amortized individual fairness subject to constraints on ranking quality as an online optimization problem and show that it can be solved as an integer linear program. Our experimental evaluation reveals that unfair attention distribution in rankings can be substantial, and demonstrates that our method can improve individual fairness while retaining high ranking quality.

arXiv commentAccepted to SIGIR 2018

References in corpus (4)

• Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatment
• Fairness of Exposure in Rankings
• FA*IR: A Fair Top-k Ranking Algorithm
• Auditing Search Engines for Differential Satisfaction Across Demographics

Cited by in corpus (67)

• Fairness of Exposure in Rankings
• Fairness in Machine Learning: A Survey
• A Survey on the Fairness of Recommender Systems
• Fairness-Aware Ranking in Search & Recommendation Systems with Application to LinkedIn Talent Search
• Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting
• Controlling Fairness and Bias in Dynamic Learning-to-Rank
• FairRec: Two-Sided Fairness for Personalized Recommendations in Two-Sided Platforms
• Evaluating Stochastic Rankings with Expected Exposure
• Deconfounded Recommendation for Alleviating Bias Amplification
• Personalized Counterfactual Fairness in Recommendation
• Building Human Values into Recommender Systems: An Interdisciplinary Synthesis
• Fighting Fire with Fire: Using Antidote Data to Improve Polarization and Fairness of Recommender Systems
• Fairness in Ranking: A Survey
• Operationalizing Individual Fairness with Pairwise Fair Representations
• A Graph-based Approach for Mitigating Multi-sided Exposure Bias in Recommender Systems
• User-controllable Recommendation Against Filter Bubbles
• Modeling Two-Way Selection Preference for Person-Job Fit
• A Stakeholder-Centered View on Fairness in Music Recommender Systems
• User Fairness, Item Fairness, and Diversity for Rankings in Two-Sided Markets
• Interplay between Upsampling and Regularization for Provider Fairness in Recommender Systems
• Societal Biases in Retrieved Contents: Measurement Framework and Adversarial Mitigation for BERT Rankers
• Equality of Learning Opportunity via Individual Fairness in Personalized Recommendations
• Rewiring What-to-Watch-Next Recommendations to Reduce Radicalization Pathways
• Do Neural Ranking Models Intensify Gender Bias?
• Evaluation Measures of Individual Item Fairness for Recommender Systems: A Critical Study
• Policy-Gradient Training of Fair and Unbiased Ranking Functions
• The few-get-richer: a surprising consequence of popularity-based rankings
• Towards Individual and Multistakeholder Fairness in Tourism Recommender Systems
• Assessing Viewpoint Diversity in Search Results Using Ranking Fairness Metrics
• Fair Ranking as Fair Division: Impact-Based Individual Fairness in Ranking
• Reaching the End of Unbiasedness: Uncovering Implicit Limitations of Click-Based Learning to Rank
• Fairness of Exposure in Light of Incomplete Exposure Estimation
• Understanding and Mitigating the Effect of Outliers in Fair Ranking
• Distributionally-Informed Recommender System Evaluation
• Evaluating Fairness in Argument Retrieval
• Are We Really Achieving Better Beyond-Accuracy Performance in Next Basket Recommendation?
• Introducing the Expohedron for Efficient Pareto-optimal Fairness-Utility Amortizations in Repeated Rankings
• Recommending With, Not For: Co-Designing Recommender Systems for Social Good
• Probabilistic Permutation Graph Search: Black-Box Optimization for Fairness in Ranking
• The Role of Relevance in Fair Ranking
• Causality-Inspired Fair Representation Learning for Multimodal Recommendation
• Predictive Uncertainty-based Bias Mitigation in Ranking
• Regulating Group Exposure for Item Providers in Recommendation
• Leave No One Behind: Fairness-Aware Cross-Domain Recommender Systems for Non-Overlapping Users
• Properties of Group Fairness Metrics for Rankings
• Can We Trust Recommender System Fairness Evaluation? The Role of Fairness and Relevance
• Fairness Rising from the Ranks: HITS and PageRank on Homophilic Networks
• On the Impact of Outlier Bias on User Clicks
• FairSort: Learning to Fair Rank for Personalized Recommendations in Two-Sided Platforms
• Adaptive Fair Representation Learning for Personalized Fairness in Recommendations via Information Alignment
• Driving the Herd: Search Engines as Content Influencers
• A Ranking Approach to Fair Classification
• Pareto-Optimal Fairness-Utility Amortizations in Rankings with a DBN Exposure Model
• Maximizing Neutrality in News Ordering
• Scalable and Provably Fair Exposure Control for Large-Scale Recommender Systems
• Fairness in Ranking under Disparate Uncertainty
• What is fair? Exploring the artists' perspective on the fairness of music streaming platforms
• From Query to Conscience: The Importance of Information Retrieval in Empowering Socially Responsible Consumerism
• RAGUEL: Recourse-Aware Group Unfairness Elimination
• Language Fairness in Multilingual Information Retrieval
• A Reproducibility Study of Product-side Fairness in Bundle Recommendation
• Learning to Re-rank with Constrained Meta-Optimal Transport
• Searching Personal Collections
• Robust Reputation Independence in Ranking Systems for Multiple Sensitive Attributes
• Exposure-Based Reinforcement Learning to Rank
• Balancing Fairness and High Match Rates in Reciprocal Recommender Systems: A Nash Social Welfare Approach
• Joint Evaluation of Fairness and Relevance in Recommender Systems with Pareto Frontier