Directional Bias Amplification

Abstract

Mitigating bias in machine learning systems requires refining our understanding of bias propagation pathways: from societal structures to large-scale data to trained models to impact on society. In this work, we focus on one aspect of the problem, namely bias amplification: the tendency of models to amplify the biases present in the data they are trained on. A metric for measuring bias amplification was introduced in the seminal work by Zhao et al. (2017); however, as we demonstrate, this metric suffers from a number of shortcomings including conflating different types of bias amplification and failing to account for varying base rates of protected attributes. We introduce and analyze a new, decoupled metric for measuring bias amplification, ${\text{BiasAmp}}_{\to }$ (Directional Bias Amplification). We thoroughly analyze and discuss both the technical assumptions and normative implications of this metric. We provide suggestions about its measurement by cautioning against predicting sensitive attributes, encouraging the use of confidence intervals due to fluctuations in the fairness of models across runs, and discussing the limitations of what this metric captures. Throughout this paper, we work to provide an interrogative look at the technical measurement of bias amplification, guided by our normative ideas of what we want it to encompass. Code is located at https://github.com/princetonvisualai/directional-bias-amp

arXiv commentICML 2021

References in corpus (14)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Semantics derived automatically from language corpora contain human-like biases
• Equality of Opportunity in Supervised Learning
• Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
• Counterfactual Fairness
• Women also Snowboard: Overcoming Bias in Captioning Models
• A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle
• Interventions over Predictions: Reframing the Ethical Debate for Actuarial Risk Assessment
• Gender Bias in Neural Natural Language Processing
• Predictive Multiplicity in Classification
• Imperfect ImaGANation: Implications of GANs Exacerbating Biases on Facial Data Augmentation and Snapchat Selfie Lenses
• Variable Importance Clouds: A Way to Explore Variable Importance for the Set of Good Models
• Fair Generative Modeling via Weak Supervision
• Feature-Wise Bias Amplification

Cited by in corpus (3)

• Toward Operationalizing Pipeline-aware ML Fairness: A Research Agenda for Developing Practical Guidelines and Tools
• Modeling Techniques for Machine Learning Fairness: A Survey
• Correcting Exposure Bias for Link Recommendation