About Face: A Survey of Facial Recognition Evaluation

Abstract

We survey over 100 face datasets constructed between 1976 to 2019 of 145 million images of over 17 million subjects from a range of sources, demographics and conditions. Our historical survey reveals that these datasets are contextually informed, shaped by changes in political motivations, technological capability and current norms. We discuss how such influences mask specific practices (some of which may actually be harmful or otherwise problematic) and make a case for the explicit communication of such details in order to establish a more grounded understanding of the technology's function in the real world.

Presented at AAAI 2020 Workshop on AI Evaluation

References in corpus (4)

• Learning Face Representation from Scratch
• Performance Measures and a Data Set for Multi-Target, Multi-Camera Tracking
• Characterizing the Variability in Face Recognition Accuracy Relative to Race
• A Replication Study: Machine Learning Models Are Capable of Predicting Sexual Orientation From Facial Images

Cited by in corpus (6)

• Bias in Automated Speaker Recognition
• Real Risks of Fake Data: Synthetic Data, Diversity-Washing and Consent Circumvention
• Evaluating CLIP: Towards Characterization of Broader Capabilities and Downstream Implications
• Mitigating Dataset Harms Requires Stewardship: Lessons from 1000 Papers
• IPFed: Identity protected federated learning for user authentication
• Ordre public exceptions for algorithmic surveillance patents