Transfer of Machine Learning Fairness across Domains

Abstract

If our models are used in new or unexpected cases, do we know if they will make fair predictions? Previously, researchers developed ways to debias a model for a single problem domain. However, this is often not how models are trained and used in practice. For example, labels and demographics (sensitive attributes) are often hard to observe, resulting in auxiliary or synthetic data to be used for training, and proxies of the sensitive attribute to be used for evaluation of fairness. A model trained for one setting may be picked up and used in many others, particularly as is common with pre-training and cloud APIs. Despite the pervasiveness of these complexities, remarkably little work in the fairness literature has theoretically examined these issues. We frame all of these settings as domain adaptation problems: how can we use what we have learned in a source domain to debias in a new target domain, without directly debiasing on the target domain as if it is a completely new problem? We offer new theoretical guarantees of improving fairness across domains, and offer a modeling approach to transfer to data-sparse target domains. We give empirical results validating the theory and showing that these modeling approaches can improve fairness metrics with less data.

References in corpus (10)

• Learning Transferable Features with Deep Adaptation Networks
• Equality of Opportunity in Supervised Learning
• Domain Separation Networks
• Domain Adaptation: Learning Bounds and Algorithms
• Robustness May Be at Odds with Accuracy
• Data Decisions and Theoretical Implications when Adversarially Learning Fair Representations
• Learning Adversarially Fair and Transferable Representations
• Satisfying Real-world Goals with Dataset Constraints
• Residual Unfairness in Fair Machine Learning from Prejudiced Data
• Discriminatory Transfer

Cited by in corpus (10)

• What-is and How-to for Fairness in Machine Learning: A Survey, Reflection, and Perspective
• Fairness without the sensitive attribute via Causal Variational Autoencoder
• Towards Fair Knowledge Transfer for Imbalanced Domain Adaptation
• Fair Classification via Domain Adaptation: A Dual Adversarial Learning Approach
• Fairness Warnings and Fair-MAML: Learning Fairly with Minimal Data
• Toward a better trade-off between performance and fairness with kernel-based distribution matching
• Robust Fairness-aware Learning Under Sample Selection Bias
• On Transferability of Bias Mitigation Effects in Language Model Fine-Tuning
• Evaluating Fairness of Machine Learning Models Under Uncertain and Incomplete Information
• Fair Meta-Learning: Learning How to Learn Fairly