Unintended Selection: Persistent Qualification Rate Disparities and Interventions

Abstract

Realistically -- and equitably -- modeling the dynamics of group-level disparities in machine learning remains an open problem. In particular, we desire models that do not suppose inherent differences between artificial groups of people -- but rather endogenize disparities by appeal to unequal initial conditions of insular subpopulations. In this paper, agents each have a real-valued feature $X$ (e.g., credit score) informed by a "true" binary label $Y$ representing qualification (e.g., for a loan). Each agent alternately (1) receives a binary classification label $\hat{Y}$ (e.g., loan approval) from a Bayes-optimal machine learning classifier observing $X$ and (2) may update their qualification $Y$ by imitating successful strategies (e.g., seek a raise) within an isolated group $G$ of agents to which they belong. We consider the disparity of qualification rates $\mathrm{Pr}(Y=1)$ between different groups and how this disparity changes subject to a sequence of Bayes-optimal classifiers repeatedly retrained on the global population. We model the evolving qualification rates of each subpopulation (group) using the replicator equation, which derives from a class of imitation processes. We show that differences in qualification rates between subpopulations can persist indefinitely for a set of non-trivial equilibrium states due to uniformed classifier deployments, even when groups are identical in all aspects except initial qualification densities. We next simulate the effects of commonly proposed fairness interventions on this dynamical system along with a new feedback control mechanism capable of permanently eliminating group-level qualification rate disparities. We conclude by discussing the limitations of our model and findings and by outlining potential future work.

arXiv comment39 pages, 10 figures, to be published in the Thirty-fifth Conference on Neural Information Processing Systems (NeurIPS 2021)

References in corpus (11)

• Counterfactual Fairness
• The Measure and Mismeasure of Fairness
• How Algorithmic Confounding in Recommendation Systems Increases Homogeneity and Decreases Utility
• The Disparate Effects of Strategic Manipulation
• A Short-term Intervention for Long-term Fairness in the Labor Market
• From Parity to Preference-based Notions of Fairness in Classification
• On the Long-term Impact of Algorithmic Decision Policies: Effort Unfairness and Feature Segregation through Social Learning
• How Do Fair Decisions Fare in Long-term Qualification?
• Group Retention when Using Machine Learning in Sequential Decision Making: the Interplay between User Dynamics and Fairness
• Dynamic Modeling and Equilibria in Fair Decision Making
• Bandit Learning with Delayed Impact of Actions

Cited by in corpus (1)

• What-is and How-to for Fairness in Machine Learning: A Survey, Reflection, and Perspective