Meta Balanced Network for Fair Face Recognition

Abstract

Although deep face recognition has achieved impressive progress in recent years, controversy has arisen regarding discrimination based on skin tone, questioning their deployment into real-world scenarios. In this paper, we aim to systematically and scientifically study this bias from both data and algorithm aspects. First, using the dermatologist approved Fitzpatrick Skin Type classification system and Individual Typology Angle, we contribute a benchmark called Identity Shades (IDS) database, which effectively quantifies the degree of the bias with respect to skin tone in existing face recognition algorithms and commercial APIs. Further, we provide two skin-tone aware training datasets, called BUPT-Globalface dataset and BUPT-Balancedface dataset, to remove bias in training data. Finally, to mitigate the algorithmic bias, we propose a novel meta-learning algorithm, called Meta Balanced Network (MBN), which learns adaptive margins in large margin loss such that the model optimized by this loss can perform fairly across people with different skin tones. To determine the margins, our method optimizes a meta skewness loss on a clean and unbiased meta set and utilizes backward-on-backward automatic differentiation to perform a second order gradient descent step on the current margins. Extensive experiments show that MBN successfully mitigates bias and learns more balanced performance for people with different skin tones in face recognition. The proposed datasets are available at http://www.whdeng.cn/RFW/index.html.

arXiv commentAccepted by IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)

References in corpus (14)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• FaceNet: A Unified Embedding for Face Recognition and Clustering
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Unsupervised Domain Adaptation by Backpropagation
• Squeeze-and-Excitation Networks
• Deep Learning Face Representation by Joint Identification-Verification
• Learning Face Representation from Scratch
• Deep Face Recognition: A Survey
• Learning to Reweight Examples for Robust Deep Learning
• NormFace: L2 Hypersphere Embedding for Face Verification
• Meta-Weight-Net: Learning an Explicit Mapping For Sample Weighting
• On the (im)possibility of fairness
• Diversity in Faces
• Fairness GAN

Cited by in corpus (5)

• How Good is ChatGPT at Face Biometrics? A First Look into Recognition, Soft Biometrics, and Explainability
• Review of Demographic Fairness in Face Recognition
• Quality-Diversity Generative Sampling for Learning with Synthetic Data
• Benchmarking the Fairness of Image Upsampling Methods
• Generating 2D and 3D Master Faces for Dictionary Attacks with a Network-Assisted Latent Space Evolution