Controlling Fairness and Bias in Dynamic Learning-to-Rank

Abstract

Rankings are the primary interface through which many online platforms match users to items (e.g. news, products, music, video). In these two-sided markets, not only the users draw utility from the rankings, but the rankings also determine the utility (e.g. exposure, revenue) for the item providers (e.g. publishers, sellers, artists, studios). It has already been noted that myopically optimizing utility to the users, as done by virtually all learning-to-rank algorithms, can be unfair to the item providers. We, therefore, present a learning-to-rank approach for explicitly enforcing merit-based fairness guarantees to groups of items (e.g. articles by the same publisher, tracks by the same artist). In particular, we propose a learning algorithm that ensures notions of amortized group fairness, while simultaneously learning the ranking function from implicit feedback data. The algorithm takes the form of a controller that integrates unbiased estimators for both fairness and utility, dynamically adapting both as more data becomes available. In addition to its rigorous theoretical foundation and convergence guarantees, we find empirically that the algorithm is highly practical and robust.

arXiv commentFirst two authors contributed equally. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval 2020

References in corpus (13)

• Adam: A Method for Stochastic Optimization
• Fairness of Exposure in Rankings
• Equity of Attention: Amortizing Individual Fairness in Rankings
• Unbiased Learning to Rank with Unbiased Propensity Estimation
• Recommendations as Treatments: Debiasing Learning and Evaluation
• Runaway Feedback Loops in Predictive Policing
• Estimating Position Bias without Intrusive Interventions
• Online Learning to Rank in Stochastic Click Models
• Intervention Harvesting for Context-Dependent Examination-Bias Estimation
• Minimally Invasive Randomization for Collecting Unbiased Preferences from Clickthrough Logs
• Policy Learning for Fairness in Ranking
• The few-get-richer: a surprising consequence of popularity-based rankings
• Consequential Ranking Algorithms and Long-term Welfare

Cited by in corpus (51)

• A Survey on the Fairness of Recommender Systems
• Towards Long-term Fairness in Recommendation
• Clicks can be Cheating: Counterfactual Recommendation for Mitigating Clickbait Issue
• Deconfounded Recommendation for Alleviating Bias Amplification
• User-controllable Recommendation Against Filter Bubbles
• Societal Biases in Retrieved Contents: Measurement Framework and Adversarial Mitigation for BERT Rankers
• A Model-Agnostic Causal Learning Framework for Recommendation using Search Data
• Maximizing Marginal Fairness for Dynamic Learning to Rank
• CausalRec: Causal Inference for Visual Debiasing in Visually-Aware Recommendation
• Evaluation Measures of Individual Item Fairness for Recommender Systems: A Critical Study
• Policy-Gradient Training of Fair and Unbiased Ranking Functions
• Fair Ranking as Fair Division: Impact-Based Individual Fairness in Ranking
• Reaching the End of Unbiasedness: Uncovering Implicit Limitations of Click-Based Learning to Rank
• Understanding and Mitigating the Effect of Outliers in Fair Ranking
• Intersectional Two-sided Fairness in Recommendation
• FAIR: Fairness-Aware Information Retrieval Evaluation
• Are We Really Achieving Better Beyond-Accuracy Performance in Next Basket Recommendation?
• Introducing the Expohedron for Efficient Pareto-optimal Fairness-Utility Amortizations in Repeated Rankings
• Subverting Fair Image Search with Generative Adversarial Perturbations
• PoissonMat: Remodeling Matrix Factorization using Poisson Distribution and Solving the Cold Start Problem without Input Data
• The Role of Relevance in Fair Ranking
• Probabilistic Permutation Graph Search: Black-Box Optimization for Fairness in Ranking
• KL-Mat : Fair Recommender System via Information Geometry
• Marginal-Certainty-aware Fair Ranking Algorithm
• Vertical Allocation-based Fair Exposure Amortizing in Ranking
• Properties of Group Fairness Metrics for Rankings
• Subgroup Fairness in Two-Sided Markets
• On the Impact of Outlier Bias on User Clicks
• Pareto-Optimal Fairness-Utility Amortizations in Rankings with a DBN Exposure Model
• Group fairness without demographics using social networks
• Recent Advances in the Foundations and Applications of Unbiased Learning to Rank
• Recommendation Fairness: From Static to Dynamic
• FARA: Future-aware Ranking Algorithm for Fairness Optimization
• Scalable and Provably Fair Exposure Control for Large-Scale Recommender Systems
• Pareto Pairwise Ranking for Fairness Enhancement of Recommender Systems
• Achievement and Fragility of Long-term Equitability
• Fairness Through Regularization for Learning to Rank
• Effective Visualization and Analysis of Recommender Systems
• Performative Debias with Fair-exposure Optimization Driven by Strategic Agents in Recommender Systems
• Fairness in Ranking under Uncertainty
• Language Fairness in Multilingual Information Retrieval
• Learning Robust Recommender from Noisy Implicit Feedback
• Fairness-Aware Online Personalization
• Guaranteeing Accuracy and Fairness under Fluctuating User Traffic: A Bankruptcy-Inspired Re-ranking Approach
• Zipf Matrix Factorization : Matrix Factorization with Matthew Effect Reduction
• Two-sided fairness in rankings via Lorenz dominance
• Calibrating Explore-Exploit Trade-off for Fair Online Learning to Rank
• Computationally Efficient Optimization of Plackett-Luce Ranking Models for Relevance and Fairness
• LogitMat : Zeroshot Learning Algorithm for Recommender Systems without Transfer Learning or Pretrained Models
• Incentives for Item Duplication under Fair Ranking Policies
• Estimation of Fair Ranking Metrics with Incomplete Judgments