A Review of Convolutional Neural Networks for Inverse Problems in Imaging

Abstract

In this survey paper, we review recent uses of convolution neural networks (CNNs) to solve inverse problems in imaging. It has recently become feasible to train deep CNNs on large databases of images, and they have shown outstanding performance on object classification and segmentation tasks. Motivated by these successes, researchers have begun to apply CNNs to the resolution of inverse problems such as denoising, deconvolution, super-resolution, and medical image reconstruction, and they have started to report improvements over state-of-the-art methods, including sparsity-based techniques such as compressed sensing. Here, we review the recent experimental work in these areas, with a focus on the critical design decisions: Where does the training data come from? What is the architecture of the CNN? and How is the learning problem formulated and solved? We also bring together a few key theoretical papers that offer perspective on why CNNs are appropriate for inverse problems and point to some next steps in the field.

References in corpus (1)

• Deep Learning in Neural Networks: An Overview

Cited by in corpus (65)

• An overview of deep learning in medical imaging focusing on MRI
• On instabilities of deep learning in image reconstruction - Does AI come at a cost?
• CNN-Based Projected Gradient Descent for Consistent Image Reconstruction
• Neuro-memristive Circuits for Edge Computing: A review
• FISTA-Net: Learning A Fast Iterative Shrinkage Thresholding Network for Inverse Problems in Imaging
• RARE: Image Reconstruction using Deep Priors Learned without Ground Truth
• Locating earthquakes with a network of seismic stations via a deep learning method
• Can stable and accurate neural networks be computed? -- On the barriers of deep learning and Smale's 18th problem
• Diffraction tomography with a deep image prior
• Learning The Invisible: A Hybrid Deep Learning-Shearlet Framework for Limited Angle Computed Tomography
• Review: Deep Learning in Electron Microscopy
• HoloStain: Holographic virtual staining of individual biological cells
• Artificial Neural Network Approach to the Analytic Continuation Problem
• Learning Variational Data Assimilation Models and Solvers
• Artificial Neural Networks for Photonic Applications: From Algorithms to Implementation
• Deep Learning Methods for Solving Linear Inverse Problems: Research Directions and Paradigms
• CNN-Based Image Reconstruction Method for Ultrafast Ultrasound Imaging
• High-Resolution Limited-Angle Phase Tomography of Dense Layered Objects Using Deep Neural Networks
• Untrained networks for compressive lensless photography
• All-optical image denoising using a diffractive visual processor
• Combining Weighted Total Variation and Deep Image Prior for natural and medical image restoration via ADMM
• Robust Phase Unwrapping via Deep Image Prior for Quantitative Phase Imaging
• Multimodal 4DVarNets for the reconstruction of sea surface dynamics from SST-SSH synergies
• Fast Point Spread Function Modeling with Deep Learning
• xQSM: Quantitative Susceptibility Mapping with Octave Convolutional and Noise Regularized Neural Networks
• Partial Scanning Transmission Electron Microscopy with Deep Learning
• Perspective: magnetic nanoparticles in theranostic applications
• Bilevel Optimization, Deep Learning and Fractional Laplacian Regularization with Applications in Tomography
• On the interplay between physical and content priors in deep learning for computational imaging
• Improving Electron Micrograph Signal-to-Noise with an Atrous Convolutional Encoder-Decoder
• Classification of stroke using Neural Networks in Electrical Impedance Tomography
• Machine learning-based inversion of nuclear responses
• SGD-Net: Efficient Model-Based Deep Learning with Theoretical Guarantees
• Infinite-dimensional inverse problems with finite measurements
• ANN-assisted CoSaMP Algorithm for Linear Electromagnetic Imaging of Spatially Sparse Domains
• Accelerating Quantitative Susceptibility Mapping using Compressed Sensing and Deep Neural Network
• Optimal Physical Preprocessing for Example-Based Super-Resolution
• Supervised learning in Hamiltonian reconstruction from local measurements on eigenstates
• Adaptive Learning Rate Clipping Stabilizes Learning
• GRIDS-Net: Inverse shape design and identification of scatterers via geometric regularization and physics-embedded deep learning
• Algorithm-driven Advances for Scientific CT Instruments: From Model-based to Deep Learning-based Approaches
• Learning-based Image Reconstruction via Parallel Proximal Algorithm
• Ultrafast Radiographic Imaging and Tracking: An overview of instruments, methods, data, and applications
• Neural networks for classification of strokes in electrical impedance tomography on a 3D head model
• Deep Learning for Biomedical Image Reconstruction: A Survey
• Warwick Electron Microscopy Datasets
• Perceptual Compressive Sensing
• PtychoDV: Vision Transformer-Based Deep Unrolling Network for Ptychographic Image Reconstruction
• Painting baryons onto N-body simulations of galaxy clusters with image-to-image deep learning
• Machine Learning for Improved Current Density Reconstruction from 2D Vector Magnetic Images
• Weak-lensing Mass Reconstruction of Galaxy Clusters with Convolutional Neural Network
• NESTANets: Stable, accurate and efficient neural networks for analysis-sparse inverse problems
• Deep Neural Networks for Computational Optical Form Measurements
• End-to-end metasurface design for temperature imaging via broadband Planck-radiation regression
• A Multimodal Deep Network for the Reconstruction of T2W MR Images
• Synthesizing the degree of polarization uniformity from non-polarization-sensitive optical coherence tomography signals using a neural network
• Posterior Estimation Using Deep Learning: A Simulation Study of Compartmental Modeling in Dynamic PET
• Localized adversarial artifacts for compressed sensing MRI
• Global Adaptive Filtering Layer for Computer Vision
• Convolutional Neural Networks Analyzed via Inverse Problem Theory and Sparse Representations
• Representation and reconstruction of covariance operators in linear inverse problems
• SRoll3: A neural network approach to reduce large-scale systematic effects in the Planck High Frequency Instrument maps
• Advancing biological super-resolution microscopy through deep learning: a brief review
• A Statistical Framework to Investigate the Optimality of Signal-Reconstruction Methods
• Overcoming Distribution Shifts in Plug-and-Play Methods with Test-Time Training