Plug-and-Play Methods for Integrating Physical and Learned Models in Computational Imaging

Abstract

Plug-and-Play Priors (PnP) is one of the most widely-used frameworks for solving computational imaging problems through the integration of physical models and learned models. PnP leverages high-fidelity physical sensor models and powerful machine learning methods for prior modeling of data to provide state-of-the-art reconstruction algorithms. PnP algorithms alternate between minimizing a data-fidelity term to promote data consistency and imposing a learned regularizer in the form of an image denoiser. Recent highly-successful applications of PnP algorithms include bio-microscopy, computerized tomography, magnetic resonance imaging, and joint ptycho-tomography. This article presents a unified and principled review of PnP by tracing its roots, describing its major variations, summarizing main results, and discussing applications in computational imaging. We also point the way towards further developments by discussing recent results on equilibrium equations that formulate the problem associated with PnP algorithms.

References in corpus (2)

• Deep Learning in Neural Networks: An Overview
• Provable Convergence of Plug-and-Play Priors with MMSE denoisers

Cited by in corpus (21)

• Phase Retrieval: From Computational Imaging to Machine Learning
• Fast Diffusion EM: a diffusion model for blind inverse problems with application to deconvolution
• Improving Diffusion Inverse Problem Solving with Decoupled Noise Annealing
• Provable Preconditioned Plug-and-Play Approach for Compressed Sensing MRI Reconstruction
• PtychoDV: Vision Transformer-Based Deep Unrolling Network for Ptychographic Image Reconstruction
• Learning Task-Specific Strategies for Accelerated MRI
• On the Contractivity of Plug-and-Play Operators
• Monotone Lipschitz-Gradient Denoiser: Explainability of Operator Regularization Approaches Free From Lipschitz Constant Control
• Denoising Particle Beam Micrographs with Plug-and-Play Methods
• Efficient high-resolution refinement in cryo-EM with stochastic gradient descent
• Gradient Networks
• Scale-Equivariant Imaging: Self-Supervised Learning for Image Super-Resolution and Deblurring
• From Nano to Macro: Overview of the IEEE Bio Image and Signal Processing Technical Committee
• Controlled Learning of Pointwise Nonlinearities in Neural-Network-Like Architectures
• INDIGO+: A Unified INN-Guided Probabilistic Diffusion Algorithm for Blind and Non-Blind Image Restoration
• Super-Resolution with Structured Motion
• A plug-and-play approach with fast uncertainty quantification for weak lensing mass mapping
• A novel method and dataset for depth-guided image deblurring from smartphone Lidar
• Overcoming Distribution Shifts in Plug-and-Play Methods with Test-Time Training
• Equivariant plug-and-play image reconstruction
• Deep Unfolding-Aided Parameter Tuning for Plug-and-Play-Based Video Snapshot Compressive Imaging