Predictive Inequity in Object Detection

Abstract

In this work, we investigate whether state-of-the-art object detection systems have equitable predictive performance on pedestrians with different skin tones. This work is motivated by many recent examples of ML and vision systems displaying higher error rates for certain demographic groups than others. We annotate an existing large scale dataset which contains pedestrians, BDD100K, with Fitzpatrick skin tones in ranges [1-3] or [4-6]. We then provide an in-depth comparative analysis of performance between these two skin tone groupings, finding that neither time of day nor occlusion explain this behavior, suggesting this disparity is not merely the result of pedestrians in the 4-6 range appearing in more difficult scenes for detection. We investigate to what extent time of day, occlusion, and reweighting the supervised loss during training affect this predictive bias.

References in corpus (1)

• Equality of Opportunity in Supervised Learning

Cited by in corpus (7)

• Robot Rights? Let's Talk about Human Welfare Instead
• Algorithmic Injustices: Towards a Relational Ethics
• Estimating Skin Tone and Effects on Classification Performance in Dermatology Datasets
• Characterizing Bias in Classifiers using Generative Models
• Age and gender bias in pedestrian detection algorithms
• Making ethical decisions for the immersive web
• Free cyclic group actions on highly-connected $2n$-manifolds