Balanced Datasets Are Not Enough: Estimating and Mitigating Gender Bias in Deep Image Representations

Abstract

In this work, we present a framework to measure and mitigate intrinsic biases with respect to protected variables --such as gender-- in visual recognition tasks. We show that trained models significantly amplify the association of target labels with gender beyond what one would expect from biased datasets. Surprisingly, we show that even when datasets are balanced such that each label co-occurs equally with each gender, learned models amplify the association between labels and gender, as much as if data had not been balanced! To mitigate this, we adopt an adversarial approach to remove unwanted features corresponding to protected variables from intermediate representations in a deep neural network -- and provide a detailed analysis of its effectiveness. Experiments on two datasets: the COCO dataset (objects), and the imSitu dataset (actions), show reductions in gender bias amplification while maintaining most of the accuracy of the original models.

arXiv comment10 pages, 7 figures, ICCV 2019

References in corpus (11)

• Semantics derived automatically from language corpora contain human-like biases
• Equality of Opportunity in Supervised Learning
• Counterfactual Fairness
• Data Decisions and Theoretical Implications when Adversarially Learning Fair Representations
• Fairness Without Demographics in Repeated Loss Minimization
• Predictive Inequity in Object Detection
• Women also Snowboard: Overcoming Bias in Captioning Models
• Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods
• ConvNets and ImageNet Beyond Accuracy: Understanding Mistakes and Uncovering Biases
• Training with the Invisibles: Obfuscating Images to Share Safely for Learning Visual Recognition Models
• Learning to Succeed while Teaching to Fail: Privacy in Closed Machine Learning Systems

Cited by in corpus (6)

• Unshuffling Data for Improved Generalization
• Hurtful Words: Quantifying Biases in Clinical Contextual Word Embeddings
• Don't Take the Easy Way Out: Ensemble Based Methods for Avoiding Known Dataset Biases
• Exposing and Correcting the Gender Bias in Image Captioning Datasets and Models
• Maximal adversarial perturbations for obfuscation: Hiding certain attributes while preserving rest
• MABR: Multilayer Adversarial Bias Removal Without Prior Bias Knowledge