CRAFT: Cultural Russian-Oriented Dataset Adaptation for Focused Text-to-Image Generation

Abstract

Despite the fact that popular text-to-image generation models cope well with international and general cultural queries, they have a significant knowledge gap regarding individual cultures. This is due to the content of existing large training datasets collected on the Internet, which are predominantly based on Western European or American popular culture. Meanwhile, the lack of cultural adaptation of the model can lead to incorrect results, a decrease in the generation quality, and the spread of stereotypes and offensive content. In an effort to address this issue, we examine the concept of cultural code and recognize the critical importance of its understanding by modern image generation models, an issue that has not been sufficiently addressed in the research community to date. We propose the methodology for collecting and processing the data necessary to form a dataset based on the cultural code, in particular the Russian one. We explore how the collected data affects the quality of generations in the national domain and analyze the effectiveness of our approach using the Kandinsky 3.1 text-to-image model. Human evaluation results demonstrate an increase in the level of awareness of Russian culture in the model.

This is arxiv version of the paper which was accepted for the Doklady Mathematics Journal in 2024

References in corpus (14)

• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
• LAION-5B: An open large-scale dataset for training next generation image-text models
• GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models
• Training Generative Adversarial Networks with Limited Data
• MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models
• SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis
• No Classification without Representation: Assessing Geodiversity Issues in Open Data Sets for the Developing World
• Retrieval-Augmented Generation for AI-Generated Content: A Survey
• Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Bias
• Emu: Enhancing Image Generation Models Using Photogenic Needles in a Haystack
• The Dark Side of Dataset Scaling: Evaluating Racial Classification in Multimodal Models
• Bridging Cultural Nuances in Dialogue Agents through Cultural Value Surveys
• Pinpointing Why Object Recognition Performance Degrades Across Income Levels and Geographies
• FusionFrames: Efficient Architectural Aspects for Text-to-Video Generation Pipeline