Practical Compositional Fairness: Understanding Fairness in Multi-Component Recommender Systems

Abstract

How can we build recommender systems to take into account fairness? Real-world recommender systems are often composed of multiple models, built by multiple teams. However, most research on fairness focuses on improving fairness in a single model. Further, recent research on classification fairness has shown that combining multiple "fair" classifiers can still result in an "unfair" classification system. This presents a significant challenge: how do we understand and improve fairness in recommender systems composed of multiple components? In this paper, we study the compositionality of recommender fairness. We consider two recently proposed fairness ranking metrics: equality of exposure and pairwise ranking accuracy. While we show that fairness in recommendation is not guaranteed to compose, we provide theory for a set of conditions under which fairness of individual models does compose. We then present an analytical framework for both understanding whether a real system's signals can achieve compositional fairness, and improving which component would have the greatest impact on the fairness of the overall system. In addition to the theoretical results, we find on multiple datasets -- including a large-scale real-world recommender system -- that the overall system's end-to-end fairness is largely achievable by improving fairness in individual components.

arXiv commentWSDM 2021

References in corpus (13)

• Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
• Fairness of Exposure in Rankings
• FA*IR: A Fair Top-k Ranking Algorithm
• Fairness-Aware Ranking in Search & Recommendation Systems with Application to LinkedIn Talent Search
• On Fairness and Calibration
• Learning Adversarially Fair and Transferable Representations
• Evaluating Stochastic Rankings with Expected Exposure
• False Information on Web and Social Media: A Survey
• A Reductions Approach to Fair Classification
• Multisided Fairness for Recommendation
• Entire Space Multi-Task Model: An Effective Approach for Estimating Post-Click Conversion Rate
• Policy Learning for Fairness in Ranking
• The Fairness of Risk Scores Beyond Classification: Bipartite Ranking and the xAUC Metric