Hierarchical Bayesian Models to Mitigate Systematic Disparities in Prediction with Proxy Outcomes

Abstract

Label bias occurs when the outcome of interest is not directly observable and instead, modeling is performed with proxy labels. When the difference between the true outcome and the proxy label is correlated with predictors, this can yield systematic disparities in predictions for different groups of interest. We propose Bayesian hierarchical measurement models to address these issues. When strong prior information about the measurement process is available, our approach improves accuracy and helps with algorithmic fairness. If prior knowledge is limited, our approach allows assessment of the sensitivity of predictions to the unknown specifications of the measurement process. This can help practitioners gauge if enough substantive information is available to guarantee the desired accuracy and avoid disparate predictions when using proxy outcomes. We demonstrate our approach through practical examples.

References in corpus (6)

• Equality of Opportunity in Supervised Learning
• The Measure and Mismeasure of Fairness
• Identifying and Correcting Label Bias in Machine Learning
• Detecting and diagnosing prior and likelihood sensitivity with power-scaling
• Fairness Evaluation in Presence of Biased Noisy Labels
• Models as Approximations II: A Model-Free Theory of Parametric Regression

Cited by in corpus (1)

• Formalising Anti-Discrimination Law in Automated Decision Systems