Unrolled Optimization with Deep Priors

Abstract

A broad class of problems at the core of computational imaging, sensing, and low-level computer vision reduces to the inverse problem of extracting latent images that follow a prior distribution, from measurements taken under a known physical image formation model. Traditionally, hand-crafted priors along with iterative optimization methods have been used to solve such problems. In this paper we present unrolled optimization with deep priors, a principled framework for infusing knowledge of the image formation into deep networks that solve inverse problems in imaging, inspired by classical iterative methods. We show that instances of the framework outperform the state-of-the-art by a substantial margin for a wide variety of imaging problems, such as denoising, deblurring, and compressed sensing magnetic resonance imaging (MRI). Moreover, we conduct experiments that explain how the framework is best used and why it outperforms previous methods.

arXiv commentFirst two authors contributed equally

Cited by in corpus (35)

• Learning to Optimize: A Primer and A Benchmark
• Small Sample Learning in Big Data Era
• Recent Advances in Imaging Around Corners
• Blending Diverse Physical Priors with Neural Networks
• Deep Unfolding with Normalizing Flow Priors for Inverse Problems
• Solving Inverse Problems by Joint Posterior Maximization with Autoencoding Prior
• Unsupervised Deep Basis Pursuit: Learning inverse problems without ground-truth data
• Neumann Networks for Inverse Problems in Imaging
• Tuning-free Plug-and-Play Proximal Algorithm for Inverse Imaging Problems
• Analysis of Deep Complex-Valued Convolutional Neural Networks for MRI Reconstruction
• Signal retrieval with measurement system knowledge using variational generative model
• Bayesian imaging using Plug & Play priors: when Langevin meets Tweedie
• A Compound Gaussian Least Squares Algorithm and Unrolled Network for Linear Inverse Problems
• Unrolled Optimization with Deep Learning-based Priors for Phaseless Inverse Scattering Problems
• Convex Regularization Behind Neural Reconstruction
• Solving Inverse Problems by Joint Posterior Maximization with a VAE Prior
• Degrees of Freedom Analysis of Unrolled Neural Networks
• Memory-efficient Learning for High-Dimensional MRI Reconstruction
• Visual Physics: Discovering Physical Laws from Videos
• TFPnP: Tuning-free Plug-and-Play Proximal Algorithm with Applications to Inverse Imaging Problems
• Feasibility-based Fixed Point Networks
• Deep Unfolding of a Proximal Interior Point Method for Image Restoration
• Multigrid Backprojection Super-Resolution and Deep Filter Visualization
• Learning Illumination Patterns for Coded Diffraction Phase Retrieval
• Deep Unrolled Network for Video Super-Resolution
• Phase Retrieval using Expectation Consistent Signal Recovery Algorithm based on Hypernetwork
• Learning a Probabilistic Strategy for Computational Imaging Sensor Selection
• Data-Driven Design for Fourier Ptychographic Microscopy
• WARP-LCA: Efficient Convolutional Sparse Coding with Locally Competitive Algorithm
• Converged Deep Framework Assembling Principled Modules for CS-MRI
• Neurally Augmented ALISTA
• Memory-efficient Learning for Large-scale Computational Imaging -- NeurIPS deep inverse workshop
• Reconstructing unseen modalities and pathology with an efficient Recurrent Inference Machine
• Physics-based Learned Design: Optimized Coded-Illumination for Quantitative Phase Imaging
• Dirty Pixels: Towards End-to-End Image Processing and Perception