A Systematic Review of Open Datasets Used in Text-to-Image (T2I) Gen AI Model Safety

Abstract

Novel research aimed at text-to-image (T2I) generative AI safety often relies on publicly available datasets for training and evaluation, making the quality and composition of these datasets crucial. This paper presents a comprehensive review of the key datasets used in the T2I research, detailing their collection methods, compositions, semantic and syntactic diversity of prompts and the quality, coverage, and distribution of harm types in the datasets. By highlighting the strengths and limitations of the datasets, this study enables researchers to find the most relevant datasets for a use case, critically assess the downstream impacts of their work given the dataset distribution, particularly regarding model safety and ethical considerations, and also identify the gaps in dataset coverage and quality that future research may address.

Accepted for publication in IEEE Access, DOI: 10.1109/ACCESS.2025.3539933

References in corpus (11)

• Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding
• Zero-Shot Text-to-Image Generation
• BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation
• Towards Fairer Datasets: Filtering and Balancing the Distribution of the People Subtree in the ImageNet Hierarchy
• Evaluating the State-of-the-Art of End-to-End Natural Language Generation: The E2E NLG Challenge
• Multimodal datasets: misogyny, pornography, and malignant stereotypes
• Inadequacies of Large Language Model Benchmarks in the Era of Generative Artificial Intelligence
• SafeGen: Mitigating Sexually Explicit Content Generation in Text-to-Image Models
• #InsTag: Instruction Tagging for Analyzing Supervised Fine-tuning of Large Language Models
• SafetyPrompts: a Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safety
• A Survey on Multimodal Benchmarks: In the Era of Large AI Models