Diffusion Models, Image Super-Resolution And Everything: A Survey

Abstract

Diffusion Models (DMs) have disrupted the image Super-Resolution (SR) field and further closed the gap between image quality and human perceptual preferences. They are easy to train and can produce very high-quality samples that exceed the realism of those produced by previous generative methods. Despite their promising results, they also come with new challenges that need further research: high computational demands, comparability, lack of explainability, color shifts, and more. Unfortunately, entry into this field is overwhelming because of the abundance of publications. To address this, we provide a unified recount of the theoretical foundations underlying DMs applied to image SR and offer a detailed analysis that underscores the unique characteristics and methodologies within this domain, distinct from broader existing reviews in the field. This survey articulates a cohesive understanding of DM principles and explores current research avenues, including alternative input domains, conditioning techniques, guidance mechanisms, corruption spaces, and zero-shot learning approaches. By offering a detailed examination of the evolution and current trends in image SR through the lens of DMs, this survey sheds light on the existing challenges and charts potential future directions, aiming to inspire further innovation in this rapidly advancing area.

References in corpus (60)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• Hierarchical Text-Conditional Image Generation with CLIP Latents
• Diffusion Models Beat GANs on Image Synthesis
• Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding
• Score-Based Generative Modeling through Stochastic Differential Equations
• Classifier-Free Diffusion Guidance
• Cascaded Diffusion Models for High Fidelity Image Generation
• dipIQ: Blind Image Quality Assessment by Learning-to-Rank Discriminable Image Pairs
• Elucidating the Design Space of Diffusion-Based Generative Models
• DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps
• Variational Diffusion Models
• Denoising Diffusion Restoration Models
• Adding Conditional Control to Text-to-Image Diffusion Models
• Adversarial Text-to-Image Synthesis: A Review
• Predictability and Surprise in Large Generative Models
• Progressive Distillation for Fast Sampling of Diffusion Models
• Diffusion Posterior Sampling for General Noisy Inverse Problems
• Tackling the Generative Learning Trilemma with Denoising Diffusion GANs
• Understanding Diffusion Models: A Unified Perspective
• Cold Diffusion: Inverting Arbitrary Image Transforms Without Noise
• Improving Diffusion Models for Inverse Problems using Manifold Constraints
• Towards Robust Blind Face Restoration with Codebook Lookup Transformer
• Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Model
• Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Models
• Image Restoration with Mean-Reverting Stochastic Differential Equations
• Sliced Score Matching: A Scalable Approach to Density and Score Estimation
• Hitchhiker's Guide to Super-Resolution: Introduction and Recent Advances
• Knowledge Distillation in Iterative Generative Models for Improved Sampling Speed
• ResShift: Efficient Diffusion Model for Image Super-resolution by Residual Shifting
• Diffusion Models already have a Semantic Latent Space
• Learning to Efficiently Sample from Diffusion Probabilistic Models
• DiffuseVAE: Efficient, Controllable and High-Fidelity Generation from Low-Dimensional Latents
• Designing a Practical Degradation Model for Deep Blind Image Super-Resolution
• Diffusion Models for Image Restoration and Enhancement: A Comprehensive Survey
• Accelerating Diffusion Models via Early Stop of the Diffusion Process
• On the Importance of Noise Scheduling for Diffusion Models
• Subspace Diffusion Generative Models
• UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Models
• DiffusionSat: A Generative Foundation Model for Satellite Imagery
• Hierarchical Integration Diffusion Model for Realistic Image Deblurring
• Inversion by Direct Iteration: An Alternative to Denoising Diffusion for Image Restoration
• Soft Diffusion: Score Matching for General Corruptions
• Learning Fast Samplers for Diffusion Models by Differentiating Through Sample Quality
• Waving Goodbye to Low-Res: A Diffusion-Wavelet Approach for Image Super-Resolution
• DiffIR: Efficient Diffusion Model for Image Restoration
• I${}^2$SB: Image-to-Image Schrödinger Bridge
• DifFace: Blind Face Restoration with Diffused Error Contraction
• Blind Image Super-Resolution: A Survey and Beyond
• Pixel-Aware Stable Diffusion for Realistic Image Super-resolution and Personalized Stylization
• Exploiting Diffusion Prior for Real-World Image Super-Resolution
• A Comprehensive Review of Deep Learning-based Single Image Super-resolution
• Wavelet Score-Based Generative Modeling
• DWA: Differential Wavelet Amplifier for Image Super-Resolution
• ResDiff: Combining CNN and Diffusion Model for Image Super-Resolution
• SeeSR: Towards Semantics-Aware Real-World Image Super-Resolution
• Sampler Scheduler for Diffusion Models
• AddSR: Accelerating Diffusion-based Blind Super-Resolution with Adversarial Diffusion Distillation
• DiffBFR: Bootstrapping Diffusion Model Towards Blind Face Restoration
• WaveDM: Wavelet-Based Diffusion Models for Image Restoration

Cited by in corpus (6)

• Diffusion Model-Based Image Editing: A Survey
• Can AI be enabled to dynamical downscaling? A Latent Diffusion Model to mimic km-scale COSMO5.0\_CLM9 simulations
• MRI super-resolution reconstruction using efficient diffusion probabilistic model with residual shifting
• Advances in Artificial Intelligence: A Review for the Creative Industries
• Dynamic Attention-Guided Diffusion for Image Super-Resolution
• Choose Your Diffusion: Efficient and flexible ways to accelerate the diffusion model in fast high energy physics simulation