Real Risks of Fake Data: Synthetic Data, Diversity-Washing and Consent Circumvention

Abstract

Machine learning systems require representations of the real world for training and testing - they require data, and lots of it. Collecting data at scale has logistical and ethical challenges, and synthetic data promises a solution to these challenges. Instead of needing to collect photos of real people's faces to train a facial recognition system, a model creator could create and use photo-realistic, synthetic faces. The comparative ease of generating this synthetic data rather than relying on collecting data has made it a common practice. We present two key risks of using synthetic data in model development. First, we detail the high risk of false confidence when using synthetic data to increase dataset diversity and representation. We base this in the examination of a real world use-case of synthetic data, where synthetic datasets were generated for an evaluation of facial recognition technology. Second, we examine how using synthetic data risks circumventing consent for data usage. We illustrate this by considering the importance of consent to the U.S. Federal Trade Commission's regulation of data collection and affected models. Finally, we discuss how these two risks exemplify how synthetic data complicates existing governance and ethical practice; by decoupling data from those it impacts, synthetic data is prone to consolidating power away those most impacted by algorithmically-mediated harm.

References in corpus (14)

• Diffusion Models Beat GANs on Image Synthesis
• Do ImageNet Classifiers Generalize to ImageNet?
• Power to the People? Opportunities and Challenges for Participatory AI
• Towards Fairer Datasets: Filtering and Balancing the Distribution of the People Subtree in the ImageNet Hierarchy
• Privacy Preserving Face Recognition Utilizing Differential Privacy
• Extracting Training Data from Diffusion Models
• Saving Face: Investigating the Ethical Concerns of Facial Recognition Auditing
• About Face: A Survey of Facial Recognition Evaluation
• On Training Sample Memorization: Lessons from Benchmarking Generative Modeling with a Large-scale Competition
• Crowdsourcing Impacts: Exploring the Utility of Crowds for Anticipating Societal Impacts of Algorithmic Decision Making
• A causal framework for discovering and removing direct and indirect discrimination
• Machine Learning practices and infrastructures
• Towards Understanding the Interplay of Generative Artificial Intelligence and the Internet
• An Evaluation of Forensic Facial Recognition