Quality-Diversity Generative Sampling for Learning with Synthetic Data

Abstract

Generative models can serve as surrogates for some real data sources by creating synthetic training datasets, but in doing so they may transfer biases to downstream tasks. We focus on protecting quality and diversity when generating synthetic training datasets. We propose quality-diversity generative sampling (QDGS), a framework for sampling data uniformly across a user-defined measure space, despite the data coming from a biased generator. QDGS is a model-agnostic framework that uses prompt guidance to optimize a quality objective across measures of diversity for synthetically generated data, without fine-tuning the generative model. Using balanced synthetic datasets generated by QDGS, we first debias classifiers trained on color-biased shape datasets as a proof-of-concept. By applying QDGS to facial data synthesis, we prompt for desired semantic concepts, such as skin tone and age, to create an intersectional dataset with a combined blend of visual features. Leveraging this balanced data for training classifiers improves fairness while maintaining accuracy on facial recognition benchmarks. Code available at: https://github.com/Cylumn/qd-generative-sampling.

arXiv commentAccepted at AAAI 2024; 7 pages main, 12 pages total, 9 figures

References in corpus (12)

• Learning Face Representation from Scratch
• Cross-Age LFW: A Database for Studying Cross-Age Face Recognition in Unconstrained Environments
• Synthetic Data from Diffusion Models Improves ImageNet Classification
• Meta Balanced Network for Fair Face Recognition
• Fair and Diverse DPP-based Data Summarization
• The Effects of Regularization and Data Augmentation are Class Dependent
• Racial Faces in-the-Wild: Reducing Racial Bias by Information Maximization Adaptation Network
• Fair Diffusion: Instructing Text-to-Image Generation Models on Fairness
• DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generation Models
• StableRep: Synthetic Images from Text-to-Image Models Make Strong Visual Representation Learners
• Characterizing Bias in Classifiers using Generative Models
• Quality Diversity through Human Feedback: Towards Open-Ended Diversity-Driven Optimization