Denoising Diffusion Restoration Models

Abstract

Many interesting tasks in image restoration can be cast as linear inverse problems. A recent family of approaches for solving these problems uses stochastic algorithms that sample from the posterior distribution of natural images given the measurements. However, efficient solutions often require problem-specific supervised training to model the posterior, whereas unsupervised methods that are not problem-specific typically rely on inefficient iterative methods. This work addresses these issues by introducing Denoising Diffusion Restoration Models (DDRM), an efficient, unsupervised posterior sampling method. Motivated by variational inference, DDRM takes advantage of a pre-trained denoising diffusion generative model for solving any linear inverse problem. We demonstrate DDRM's versatility on several image datasets for super-resolution, deblurring, inpainting, and colorization under various amounts of measurement noise. DDRM outperforms the current leading unsupervised methods on the diverse ImageNet dataset in reconstruction quality, perceptual quality, and runtime, being 5x faster than the nearest competitor. DDRM also generalizes well for natural images out of the distribution of the observed ImageNet training set.

Project page: https://ddrm-ml.github.io/

Cited by in corpus (27)

• Diffusion Models in Vision: A Survey
• Solving 3D Inverse Problems using Pre-trained 2D Diffusion Models
• StoRM: A Diffusion-based Stochastic Regeneration Model for Speech Enhancement and Dereverberation
• CBCT-Based Synthetic CT Image Generation Using Conditional Denoising Diffusion Probabilistic Model
• Diffusion Models, Image Super-Resolution And Everything: A Survey
• SAR Despeckling using a Denoising Diffusion Probabilistic Model
• A Review on Low-Dose Emission Tomography Post-Reconstruction Denoising with Neural Network Approaches
• Artifact Reduction in 3D and 4D Cone-beam Computed Tomography Images with Deep Learning -- A Review
• PatchNR: Learning from Very Few Images by Patch Normalizing Flow Regularization
• A review of advancements in low-light image enhancement using deep learning
• Fast Diffusion EM: a diffusion model for blind inverse problems with application to deconvolution
• Conditioning and Sampling in Variational Diffusion Models for Speech Super-Resolution
• Iterative $\alpha$-(de)Blending: a Minimalist Deterministic Diffusion Model
• Solving Inverse Problems with Score-Based Generative Priors learned from Noisy Data
• Passive Non-Line-of-Sight Imaging with Light Transport Modulation
• Diffiner: A Versatile Diffusion-based Generative Refiner for Speech Enhancement
• DRUM: Diffusion-based runoff model for probabilistic flood forecasting
• Diffusion-Based Audio Inpainting
• CoNeS: Conditional neural fields with shift modulation for multi-sequence MRI translation
• Equivariant Bootstrapping for Uncertainty Quantification in Imaging Inverse Problems
• Adaptive and Iterative Point Cloud Denoising with Score-Based Diffusion Model
• Score-based Diffusion Models for Bayesian Image Reconstruction
• Iterative Window Mean Filter: Thwarting Diffusion-based Adversarial Purification
• Monte Carlo guided Diffusion for Bayesian linear inverse problems
• Adapt and Diffuse: Sample-adaptive Reconstruction via Latent Diffusion Models
• Adaptive Image Restoration for Video Surveillance: A Real-Time Approach
• Equivariant plug-and-play image reconstruction