Selective Regression Under Fairness Criteria

Abstract

Selective regression allows abstention from prediction if the confidence to make an accurate prediction is not sufficient. In general, by allowing a reject option, one expects the performance of a regression model to increase at the cost of reducing coverage (i.e., by predicting on fewer samples). However, as we show, in some cases, the performance of a minority subgroup can decrease while we reduce the coverage, and thus selective regression can magnify disparities between different sensitive subgroups. Motivated by these disparities, we propose new fairness criteria for selective regression requiring the performance of every subgroup to improve with a decrease in coverage. We prove that if a feature representation satisfies the sufficiency criterion or is calibrated for mean and variance, than the proposed fairness criteria is met. Further, we introduce two approaches to mitigate the performance disparity across subgroups: (a) by regularizing an upper bound of conditional mutual information under a Gaussian assumption and (b) by regularizing a contrastive loss for conditional mean and conditional variance prediction. The effectiveness of these approaches is demonstrated on synthetic and real-world datasets.

References in corpus (8)

• Equality of Opportunity in Supervised Learning
• Meta-learners for Estimating Heterogeneous Treatment Effects using Machine Learning
• Invariant Risk Minimization
• On Fairness and Calibration
• AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Bias
• SelectiveNet: A Deep Neural Network with an Integrated Reject Option
• CLUB: A Contrastive Log-ratio Upper Bound of Mutual Information
• Evaluation of Neural Architectures Trained with Square Loss vs Cross-Entropy in Classification Tasks