A Review on Deep Learning in Medical Image Reconstruction

Abstract

Medical imaging is crucial in modern clinics to guide the diagnosis and treatment of diseases. Medical image reconstruction is one of the most fundamental and important components of medical imaging, whose major objective is to acquire high-quality medical images for clinical usage at minimal cost and risk to the patients. Mathematical models in medical image reconstruction or, more generally, image restoration in computer vision, have been playing a prominent role. Earlier mathematical models are mostly designed by human knowledge or hypothesis on the image to be reconstructed, and we shall call these models handcrafted models. Later, handcrafted plus data-driven modeling started to emerge which still mostly relies on human designs, while part of the model is learned from the observed data. More recently, as more data and computation resources are made available, deep learning based models (or deep models) pushed data-driven modeling to the extreme where the models are mostly based on learning with minimal human designs. Both handcrafted and data-driven modeling have their own advantages and disadvantages. One of the major research trends in medical imaging is to combine handcrafted modeling with deep modeling so that we can enjoy benefits from both approaches. The major part of this article is to provide a conceptual review of some recent works on deep modeling from the unrolling dynamics viewpoint. This viewpoint stimulates new designs of neural network architectures with inspiration from optimization algorithms and numerical differential equations. Given the popularity of deep modeling, there are still vast remaining challenges in the field, as well as opportunities which we shall discuss at the end of this article.

arXiv comment31 pages, 6 figures. Survey paper. Revise the typos

References in corpus (46)

• Robust Recovery of Subspace Structures by Low-Rank Representation
• Deep Convolutional Neural Network for Inverse Problems in Imaging
• Low-Dose CT with a Residual Encoder-Decoder Convolutional Neural Network (RED-CNN)
• Image reconstruction by domain transform manifold learning
• Low Dose CT Image Denoising Using a Generative Adversarial Network with Wasserstein Distance and Perceptual Loss
• Image Restoration Using Very Deep Convolutional Encoder-Decoder Networks with Symmetric Skip Connections
• Learned Primal-dual Reconstruction
• PDE-Net 2.0: Learning PDEs from Data with A Numeric-Symbolic Hybrid Deep Network
• Solving ill-posed inverse problems using iterative deep neural networks
• A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights
• Shake-Shake regularization
• Mini-Batch Semi-Stochastic Gradient Descent in the Proximal Setting
• Universal Function Approximation by Deep Neural Nets with Bounded Width and ReLU Activations
• The Reversible Residual Network: Backpropagation Without Storing Activations
• Deep Residual Learning for Compressed Sensing CT Reconstruction via Persistent Homology Analysis
• Why Deep Neural Networks for Function Approximation?
• Theoretical Linear Convergence of Unfolded ISTA and its Practical Weights and Thresholds
• ResNet with one-neuron hidden layers is a Universal Approximator
• Optimal approximation of continuous functions by very deep ReLU networks
• Approximating Continuous Functions by ReLU Nets of Minimal Width
• Representation Benefits of Deep Feedforward Networks
• MgNet: A Unified Framework of Multigrid and Convolutional Neural Network
• A Lyapunov Analysis of Momentum Methods in Optimization
• Understanding and Improving Transformer From a Multi-Particle Dynamic System Point of View
• The power of deeper networks for expressing natural functions
• A Priori Estimates of the Population Risk for Two-layer Neural Networks
• Learning Functions: When Is Deep Better Than Shallow
• ADMM-Net: A Deep Learning Approach for Compressive Sensing MRI
• Nonlinear Approximation via Compositions
• You Only Propagate Once: Accelerating Adversarial Training via Maximal Principle
• A Mean-Field Optimal Control Formulation of Deep Learning
• Convolutional Neural Networks combined with Runge-Kutta Methods
• A Priori Estimates of the Population Risk for Residual Networks
• Deep Residual Learning and PDEs on Manifold
• Nonlocal Neural Networks, Nonlocal Diffusion and Nonlocal Modeling
• A view of Neural Networks as dynamical systems
• Differentiable Linearized ADMM
• When Image Denoising Meets High-Level Vision Tasks: A Deep Learning Approach
• Dynamically Unfolding Recurrent Restorer: A Moving Endpoint Control Method for Image Restoration
• Connecting Image Denoising and High-Level Vision Tasks via Deep Learning
• Stochastic Training of Residual Networks: a Differential Equation Viewpoint
• Multiscale Adaptive Representation of Signals: I. The Basic Framework
• ResNets Ensemble via the Feynman-Kac Formalism to Improve Natural and Robust Accuracies
• Towards Robust ResNet: A Small Step but A Giant Leap
• On the Convergence of Learning-based Iterative Methods for Nonconvex Inverse Problems
• End-to-end Lung Nodule Detection in Computed Tomography

Cited by in corpus (5)

• Deep Learning for Biomedical Image Reconstruction: A Survey
• NESTANets: Stable, accurate and efficient neural networks for analysis-sparse inverse problems
• Generative Adversarial Networks (GAN) Powered Fast Magnetic Resonance Imaging -- Mini Review, Comparison and Perspectives
• Learning to Scan: A Deep Reinforcement Learning Approach for Personalized Scanning in CT Imaging
• Improving Generalizability in Limited-Angle CT Reconstruction with Sinogram Extrapolation