The Chosen One: Consistent Characters in Text-to-Image Diffusion Models

Abstract

Recent advances in text-to-image generation models have unlocked vast potential for visual creativity. However, the users that use these models struggle with the generation of consistent characters, a crucial aspect for numerous real-world applications such as story visualization, game development, asset design, advertising, and more. Current methods typically rely on multiple pre-existing images of the target character or involve labor-intensive manual processes. In this work, we propose a fully automated solution for consistent character generation, with the sole input being a text prompt. We introduce an iterative procedure that, at each stage, identifies a coherent set of images sharing a similar identity and extracts a more consistent identity from this set. Our quantitative analysis demonstrates that our method strikes a better balance between prompt alignment and identity consistency compared to the baseline methods, and these findings are reinforced by a user study. To conclude, we showcase several practical applications of our approach.

Accepted to SIGGRAPH 2024. Project page is available at https://omriavrahami.com/the-chosen-one/

References in corpus (7)

• LoRA: Low-Rank Adaptation of Large Language Models
• Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding
• Blended Latent Diffusion
• eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers
• SpaText: Spatio-Textual Representation for Controllable Image Generation
• KNN-Diffusion: Image Generation via Large-Scale Retrieval
• Dreamix: Video Diffusion Models are General Video Editors

Cited by in corpus (1)

• Stable Flow: Vital Layers for Training-Free Image Editing