Learning Proximal Operators: Using Denoising Networks for Regularizing Inverse Imaging Problems

Abstract

While variational methods have been among the most powerful tools for solving linear inverse problems in imaging, deep (convolutional) neural networks have recently taken the lead in many challenging benchmarks. A remaining drawback of deep learning approaches is their requirement for an expensive retraining whenever the specific problem, the noise level, noise type, or desired measure of fidelity changes. On the contrary, variational methods have a plug-and-play nature as they usually consist of separate data fidelity and regularization terms. In this paper we study the possibility of replacing the proximal operator of the regularization used in many convex energy minimization algorithms by a denoising neural network. The latter therefore serves as an implicit natural image prior, while the data term can still be chosen independently. Using a fixed denoising neural network in exemplary problems of image deconvolution with different blur kernels and image demosaicking, we obtain state-of-the-art reconstruction results. These indicate the high generalizability of our approach and a reduction of the need for problem-specific training. Additionally, we discuss novel results on the analysis of possible optimization algorithms to incorporate the network into, as well as the choices of algorithm parameters and their relation to the noise level the neural network is trained on.

Cited by in corpus (36)

• Deep Learning for Single Image Super-Resolution: A Brief Review
• Learned Primal-dual Reconstruction
• Denoising Prior Driven Deep Neural Network for Image Restoration
• Plug-and-Play Methods for Integrating Physical and Learned Models in Computational Imaging
• An Online Plug-and-Play Algorithm for Regularized Image Reconstruction
• RARE: Image Reconstruction using Deep Priors Learned without Ground Truth
• WINNet: Wavelet-inspired Invertible Network for Image Denoising
• Learning The Invisible: A Hybrid Deep Learning-Shearlet Framework for Limited Angle Computed Tomography
• Regularization by architecture: A deep prior approach for inverse problems
• A Partially Learned Algorithm for Joint Photoacoustic Reconstruction and Segmentation
• Provable Convergence of Plug-and-Play Priors with MMSE denoisers
• SIMBA: Scalable Inversion in Optical Tomography using Deep Denoising Priors
• Deep Iterative Reconstruction for Phase Retrieval
• Fixed-Point and Objective Convergence of Plug-and-Play Algorithms
• Plug-and-play ISTA converges with kernel denoisers
• Regularized Fourier Ptychography using an Online Plug-and-Play Algorithm
• Systematic Review on Learning-based Spectral CT
• Solving Inverse Problems by Joint Posterior Maximization with Autoencoding Prior
• Rotation Equivariant Proximal Operator for Deep Unfolding Methods in Image Restoration
• Wasserstein Patch Prior for Image Superresolution
• A New Recurrent Plug-and-Play Prior Based on the Multiple Self-Similarity Network
• On Maximum-a-Posteriori estimation with Plug & Play priors and stochastic gradient descent
• On the Proof of Fixed-Point Convergence for Plug-and-Play ADMM
• Provable Preconditioned Plug-and-Play Approach for Compressed Sensing MRI Reconstruction
• Deep Model-Based Super-Resolution with Non-uniform Blur
• Plug-and-Play Regularization using Linear Solvers
• Fast and Robust Cascade Model for Multiple Degradation Single Image Super-Resolution
• A Compound Gaussian Least Squares Algorithm and Unrolled Network for Linear Inverse Problems
• Local Kernels that Approximate Bayesian Regularization and Proximal Operators
• Unrolled Optimization with Deep Learning-based Priors for Phaseless Inverse Scattering Problems
• Supervision by Denoising for Medical Image Segmentation
• A plug-and-play framework for curvilinear structure segmentation based on a learned reconnecting regularization
• Monotone Lipschitz-Gradient Denoiser: Explainability of Operator Regularization Approaches Free From Lipschitz Constant Control
• Training Auto-encoder-based Optimizers for Terahertz Image Reconstruction
• Provably Convergent Plug & Play Linearized ADMM, applied to Deblurring Spatially Varying Kernels
• Stability Bounds for the Unfolded Forward-Backward Algorithm