Multimodal datasets: misogyny, pornography, and malignant stereotypes

Abstract

We have now entered the era of trillion parameter machine learning models trained on billion-sized datasets scraped from the internet. The rise of these gargantuan datasets has given rise to formidable bodies of critical work that has called for caution while generating these large datasets. These address concerns surrounding the dubious curation practices used to generate these datasets, the sordid quality of alt-text data available on the world wide web, the problematic content of the CommonCrawl dataset often used as a source for training large language models, and the entrenched biases in large-scale visio-linguistic models (such as OpenAI's CLIP model) trained on opaque datasets (WebImageText). In the backdrop of these specific calls of caution, we examine the recently released LAION-400M dataset, which is a CLIP-filtered dataset of Image-Alt-text pairs parsed from the Common-Crawl dataset. We found that the dataset contains, troublesome and explicit images and text pairs of rape, pornography, malign stereotypes, racist and ethnic slurs, and other extremely problematic content. We outline numerous implications, concerns and downstream harms regarding the current state of large scale datasets while raising open questions for various stakeholders including the AI community, regulators, policy makers and data subjects.

arXiv comment33 pages

References in corpus (17)

• Learning Transferable Visual Models From Natural Language Supervision
• Language Models are Few-Shot Learners
• YFCC100M: The New Data in Multimedia Research
• Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision
• Zero-Shot Text-to-Image Generation
• The Pile: An 800GB Dataset of Diverse Text for Language Modeling
• WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learning
• Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets
• AI and Accessibility: A Discussion of Ethical Considerations
• Tencent ML-Images: A Large-Scale Multi-Label Image Database for Visual Representation Learning
• A Study of Face Obfuscation in ImageNet
• Data and its (dis)contents: A survey of dataset development and use in machine learning research
• Against Scale: Provocations and Resistances to Scale Thinking
• Computer Vision and Conflicting Values: Describing People with Automated Alt Text
• CLUECorpus2020: A Large-scale Chinese Corpus for Pre-training Language Model
• Detoxifying Language Models Risks Marginalizing Minority Voices
• Reading Isn't Believing: Adversarial Attacks On Multi-Modal Neurons

Cited by in corpus (13)

• Generative AI
• Opening up ChatGPT: Tracking openness, transparency, and accountability in instruction-tuned text generators
• AI's Regimes of Representation: A Community-centered Study of Text-to-Image Models in South Asia
• A Survey of AI Text-to-Image and AI Text-to-Video Generators
• Data Governance in the Age of Large-Scale Data-Driven Language Technology
• The Entoptic Field Camera as Metaphor-Driven Research-through-Design with AI Technologies
• Exploiting Cultural Biases via Homoglyphs in Text-to-Image Synthesis
• 'Person' == Light-skinned, Western Man, and Sexualization of Women of Color: Stereotypes in Stable Diffusion
• Responding to Generative AI Technologies with Research-through-Design: The Ryelands AI Lab as an Exploratory Study
• Evaluating Biased Attitude Associations of Language Models in an Intersectional Context
• Evaluating the Fairness of Discriminative Foundation Models in Computer Vision
• VLSlice: Interactive Vision-and-Language Slice Discovery
• A Systematic Review of Open Datasets Used in Text-to-Image (T2I) Gen AI Model Safety