SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis

Abstract

We present SDXL, a latent diffusion model for text-to-image synthesis. Compared to previous versions of Stable Diffusion, SDXL leverages a three times larger UNet backbone: The increase of model parameters is mainly due to more attention blocks and a larger cross-attention context as SDXL uses a second text encoder. We design multiple novel conditioning schemes and train SDXL on multiple aspect ratios. We also introduce a refinement model which is used to improve the visual fidelity of samples generated by SDXL using a post-hoc image-to-image technique. We demonstrate that SDXL shows drastically improved performance compared the previous versions of Stable Diffusion and achieves results competitive with those of black-box state-of-the-art image generators. In the spirit of promoting open research and fostering transparency in large model training and evaluation, we provide access to code and model weights at https://github.com/Stability-AI/generative-models

Cited by in corpus (16)

• Towards Small Object Editing: A Benchmark Dataset and A Training-Free Approach
• Diffusion Model-Based Image Editing: A Survey
• General Purpose Artificial Intelligence Systems (GPAIS): Properties, Definition, Taxonomy, Societal Implications and Responsible Governance
• How People Prompt to Create Interactive VR Scenes
• Pixel super-resolved virtual staining of label-free tissue using diffusion models
• Latent Denoising Diffusion GAN: Faster sampling, Higher image quality
• RadEdit: stress-testing biomedical vision models via diffusion image editing
• ElasticDiffusion: Training-free Arbitrary Size Image Generation through Global-Local Content Separation
• Ship in Sight: Diffusion Models for Ship-Image Super Resolution
• Tackling Copyright Issues in AI Image Generation Through Originality Estimation and Genericization
• fruit-SALAD: A Style Aligned Artwork Dataset to reveal similarity perception in image embeddings
• From Pampas to Pixels: Fine-Tuning Diffusion Models for Gaúcho Heritage
• IgCONDA-PET: Weakly-Supervised PET Anomaly Detection using Implicitly-Guided Attention-Conditional Counterfactual Diffusion Modeling -- a Multi-Center, Multi-Cancer, and Multi-Tracer Study
• CRAFT: Cultural Russian-Oriented Dataset Adaptation for Focused Text-to-Image Generation
• GABInsight: Exploring Gender-Activity Binding Bias in Vision-Language Models
• Fidelity-preserving enhancement of ptychography with foundational text-to-image models