Equivariant plug-and-play image reconstruction

Abstract

Plug-and-play algorithms constitute a popular framework for solving inverse imaging problems that rely on the implicit definition of an image prior via a denoiser. These algorithms can leverage powerful pre-trained denoisers to solve a wide range of imaging tasks, circumventing the necessity to train models on a per-task basis. Unfortunately, plug-and-play methods often show unstable behaviors, hampering their promise of versatility and leading to suboptimal quality of reconstructed images. In this work, we show that enforcing equivariance to certain groups of transformations (rotations, reflections, and/or translations) on the denoiser strongly improves the stability of the algorithm as well as its reconstruction quality. We provide a theoretical analysis that illustrates the role of equivariance on better performance and stability. We present a simple algorithm that enforces equivariance on any existing denoiser by simply applying a random transformation to the input of the denoiser and the inverse transformation to the output at each iteration of the algorithm. Experiments on multiple imaging modalities and denoising networks show that the equivariant plug-and-play algorithm improves both the reconstruction performance and the stability compared to their non-equivariant counterparts.

References in corpus (16)

• Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising
• Denoising Diffusion Probabilistic Models
• Deep Convolutional Neural Network for Inverse Problems in Imaging
• Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges
• fastMRI: An Open Dataset and Benchmarks for Accelerated MRI
• SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks
• Plug and play methods for magnetic resonance imaging (long version)
• Denoising Diffusion Restoration Models
• Plug-and-Play Methods for Integrating Physical and Learned Models in Computational Imaging
• Diffusion Posterior Sampling for General Noisy Inverse Problems
• Image reconstruction algorithms in radio interferometry: from handcrafted to learned regularization denoisers
• Gradient Step Denoiser for convergent Plug-and-Play
• Rotation-Equivariant Deep Learning for Diffusion MRI
• Sensing Theorems for Unsupervised Learning in Linear Inverse Problems
• Bayesian imaging using Plug & Play priors: when Langevin meets Tweedie
• 4D Panoptic Segmentation as Invariant and Equivariant Field Prediction