Multi-Target Multiplicity: Flexibility and Fairness in Target Specification under Resource Constraints

Abstract

Prediction models have been widely adopted as the basis for decision-making in domains as diverse as employment, education, lending, and health. Yet, few real world problems readily present themselves as precisely formulated prediction tasks. In particular, there are often many reasonable target variable options. Prior work has argued that this is an important and sometimes underappreciated choice, and has also shown that target choice can have a significant impact on the fairness of the resulting model. However, the existing literature does not offer a formal framework for characterizing the extent to which target choice matters in a particular task. Our work fills this gap by drawing connections between the problem of target choice and recent work on predictive multiplicity. Specifically, we introduce a conceptual and computational framework for assessing how the choice of target affects individuals' outcomes and selection rate disparities across groups. We call this multi-target multiplicity. Along the way, we refine the study of single-target multiplicity by introducing notions of multiplicity that respect resource constraints -- a feature of many real-world tasks that is not captured by existing notions of predictive multiplicity. We apply our methods on a healthcare dataset, and show that the level of multiplicity that stems from target variable choice can be greater than that stemming from nearly-optimal models of a single target.

Conference on Fairness, Accountability, and Transparency (FAccT '23), June 12-15, 2023, Chicago, IL, USA

References in corpus (2)

• Classifier Technology and the Illusion of Progress
• Problem Formulation and Fairness

Cited by in corpus (6)

• Fairness and Bias in Algorithmic Hiring: a Multidisciplinary Survey
• One Model Many Scores: Using Multiverse Analysis to Prevent Fairness Hacking and Evaluate the Influence of Model Design Decisions
• What Constitutes a Less Discriminatory Algorithm?
• Allocation Multiplicity: Evaluating the Promises of the Rashomon Set
• Understanding trade-offs in classifier bias with quality-diversity optimization: an application to talent management
• Falsifying Discriminant Validity of Predictive Algorithms