MoDL: Model Based Deep Learning Architecture for Inverse Problems

Abstract

We introduce a model-based image reconstruction framework with a convolution neural network (CNN) based regularization prior. The proposed formulation provides a systematic approach for deriving deep architectures for inverse problems with the arbitrary structure. Since the forward model is explicitly accounted for, a smaller network with fewer parameters is sufficient to capture the image information compared to black-box deep learning approaches, thus reducing the demand for training data and training time. Since we rely on end-to-end training, the CNN weights are customized to the forward model, thus offering improved performance over approaches that rely on pre-trained denoisers. The main difference of the framework from existing end-to-end training strategies is the sharing of the network weights across iterations and channels. Our experiments show that the decoupling of the number of iterations from the network complexity offered by this approach provides benefits including lower demand for training data, reduced risk of overfitting, and implementations with significantly reduced memory footprint. We propose to enforce data-consistency by using numerical optimization blocks such as conjugate gradients algorithm within the network; this approach offers faster convergence per iteration, compared to methods that rely on proximal gradients steps to enforce data consistency. Our experiments show that the faster convergence translates to improved performance, especially when the available GPU memory restricts the number of iterations.

published in IEEE Transaction on Medical Imaging

References in corpus (1)

• ${\ell }_0$ Sparsifying Transform Learning with Efficient Optimal Updates and Convergence Guarantees

Cited by in corpus (104)

• Self-Supervised Learning of Physics-Guided Reconstruction Neural Networks without Fully-Sampled Reference Data
• FISTA-Net: Learning A Fast Iterative Shrinkage Thresholding Network for Inverse Problems in Imaging
• Results of the 2020 fastMRI Challenge for Machine Learning MR Image Reconstruction
• Advancing machine learning for MR image reconstruction with an open competition: Overview of the 2019 fastMRI challenge
• Adaptive Diffusion Priors for Accelerated MRI Reconstruction
• RARE: Image Reconstruction using Deep Priors Learned without Ground Truth
• Solving Inverse Problems With Deep Neural Networks -- Robustness Included?
• Memory-Augmented Deep Unfolding Network for Compressive Sensing
• J-MoDL: Joint Model-Based Deep Learning for Optimized Sampling and Reconstruction
• Dense Recurrent Neural Networks for Accelerated MRI: History-Cognizant Unrolling of Optimization Algorithms
• High-Frequency Space Diffusion Models for Accelerated MRI
• Bayesian MRI Reconstruction with Joint Uncertainty Estimation using Diffusion Models
• B-spline Parameterized Joint Optimization of Reconstruction and K-space Trajectories (BJORK) for Accelerated 2D MRI
• Self-Supervised Physics-Based Deep Learning MRI Reconstruction Without Fully-Sampled Data
• MoDL-MUSSELS: Model-Based Deep Learning for Multi-Shot Sensitivity Encoded Diffusion MRI
• Deep Iterative Reconstruction for Phase Retrieval
• pISTA-SENSE-ResNet for Parallel MRI Reconstruction
• Multimodal 4DVarNets for the reconstruction of sea surface dynamics from SST-SSH synergies
• Multi-Mask Self-Supervised Learning for Physics-Guided Neural Networks in Highly Accelerated MRI
• MRI Reconstruction Using Deep Bayesian Estimation
• PUERT: Probabilistic Under-sampling and Explicable Reconstruction Network for CS-MRI
• Accelerated Coronary MRI with sRAKI: A Database-Free Self-Consistent Neural Network k-space Reconstruction for Arbitrary Undersampling
• Neural Networks-based Regularization for Large-Scale Medical Image Reconstruction
• Physics-based Reconstruction Methods for Magnetic Resonance Imaging
• Scan Specific Artifact Reduction in K-space (SPARK) Neural Networks Synergize with Physics-based Reconstruction to Accelerate MRI
• SGD-Net: Efficient Model-Based Deep Learning with Theoretical Guarantees
• An End-To-End-Trainable Iterative Network Architecture for Accelerated Radial Multi-Coil 2D Cine MR Image Reconstruction
• An Unsupervised Deep Learning Method for Multi-coil Cine MRI
• Self-Supervised Physics-Guided Deep Learning Reconstruction For High-Resolution 3D LGE CMR
• LADMM-Net: An Unrolled Deep Network For Spectral Image Fusion From Compressive Data
• Neural Architecture Search for Compressed Sensing Magnetic Resonance Image Reconstruction
• High-resolution Multi-spectral Imaging with Diffractive Lenses and Learned Reconstruction
• Unified Supervised-Unsupervised (SUPER) Learning for X-ray CT Image Reconstruction
• Fast Low Rank column-wise Compressive Sensing for Accelerated Dynamic MRI
• Joint Frequency and Image Space Learning for MRI Reconstruction and Analysis
• Computational MRI with Physics-based Constraints: Application to Multi-contrast and Quantitative Imaging
• ENSURE: A General Approach for Unsupervised Training of Deep Image Reconstruction Algorithms
• Joint Deep Reversible Regression Model and Physics-Informed Unsupervised Learning for Temperature Field Reconstruction
• Data-Consistent Non-Cartesian Deep Subspace Learning for Efficient Dynamic MR Image Reconstruction
• Zero-DeepSub: Zero-Shot Deep Subspace Reconstruction for Rapid Multiparametric Quantitative MRI Using 3D-QALAS
• Linear-Model-inspired Neural Network for Electromagnetic Inverse Scattering
• An Interpretable MRI Reconstruction Network with Two-grid-cycle Correction and Geometric Prior Distillation
• Deep, Deep Learning with BART
• Physics-Coupled Neural Network Magnetic Resonance Electrical Property Tomography (MREPT) for Conductivity Reconstruction
• Efficient approximation of Jacobian matrices involving a non-uniform fast Fourier transform (NUFFT)
• Momentum-Net: Fast and convergent iterative neural network for inverse problems
• Fast Hierarchical Deep Unfolding Network for Image Compressed Sensing
• PINQI: An End-to-End Physics-Informed Approach to Learned Quantitative MRI Reconstruction
• Assessment of Data Consistency through Cascades of Independently Recurrent Inference Machines for fast and robust accelerated MRI reconstruction
• Provable Preconditioned Plug-and-Play Approach for Compressed Sensing MRI Reconstruction
• Artificial Intelligence for Neuro MRI Acquisition: A Review
• A deep cascade of ensemble of dual domain networks with gradient-based T1 assistance and perceptual refinement for fast MRI reconstruction
• Data and Physics driven Deep Learning Models for Fast MRI Reconstruction: Fundamentals and Methodologies
• Nest-DGIL: Nesterov-optimized Deep Geometric Incremental Learning for CS Image Reconstruction
• AutoSamp: Autoencoding k-space Sampling via Variational Information Maximization for 3D MRI
• An untrained deep learning method for reconstructing dynamic magnetic resonance images from accelerated model-based data
• PtychoDV: Vision Transformer-Based Deep Unrolling Network for Ptychographic Image Reconstruction
• Uniformly convex neural networks and non-stationary iterated network Tikhonov (iNETT) method
• T2LR-Net: An unrolling network learning transformed tensor low-rank prior for dynamic MR image reconstruction
• Recursive Deformable Image Registration Network with Mutual Attention
• Fast MRI Reconstruction via Edge Attention
• GroupCDL: Interpretable Denoising and Compressed Sensing MRI via Learned Group-Sparsity and Circulant Attention
• Physics-informed self-supervised deep learning reconstruction for accelerated first-pass perfusion cardiac MRI
• A Complex Quasi-Newton Proximal Method for Image Reconstruction in Compressed Sensing MRI
• Self-Supervised Learning for Improved Calibrationless Radial MRI with NLINV-Net
• Efficient Physics-Based Learned Reconstruction Methods for Real-Time 3D Near-Field MIMO Radar Imaging
• A Tutorial on MRI Reconstruction: From Modern Methods to Clinical Implications
• Plug-and-Play Latent Feature Editing for Orientation-Adaptive Quantitative Susceptibility Mapping Neural Networks
• Motion-robust free-running volumetric cardiovascular MRI
• Learning Task-Specific Strategies for Accelerated MRI
• Memory Efficient Model Based Deep Learning Reconstructions for High Spatial Resolution 3D Non-Cartesian Acquisitions
• High-efficient Bloch simulation of magnetic resonance imaging sequences based on deep learning
• Latent Signal Models: Learning Compact Representations of Signal Evolution for Improved Time-Resolved, Multi-contrast MRI
• Robust multi-coil MRI reconstruction via self-supervised denoising
• High-Fidelity Accelerated MRI Reconstruction by Scan-Specific Fine-Tuning of Physics-Based Neural Networks
• MRI Reconstruction with Regularized 3D Diffusion Model (R3DM)
• Solving the Wide-band Inverse Scattering Problem via Equivariant Neural Networks
• Sparsity-Driven Parallel Imaging Consistency for Improved Self-Supervised MRI Reconstruction
• Optimal MRI Undersampling Patterns for Ultimate Benefit of Medical Vision Tasks
• Deep MRI Reconstruction with Radial Subsampling
• mcLARO: Multi-Contrast Learned Acquisition and Reconstruction Optimization for simultaneous quantitative multi-parametric mapping
• Recovering high-quality FODs from a reduced number of diffusion-weighted images using a model-driven deep learning architecture
• NoSENSE: Learned unrolled cardiac MRI reconstruction without explicit sensitivity maps
• Artificial Intelligence-Based Image Reconstruction in Cardiac Magnetic Resonance
• JotlasNet: Joint Tensor Low-Rank and Attention-based Sparse Unrolling Network for Accelerating Dynamic MRI
• AliasNet: Alias Artefact Suppression Network for Accelerated Phase-Encode MRI
• Improved Supervised Training of Physics-Guided Deep Learning Image Reconstruction with Multi-Masking
• Deep-ER: Deep Learning ECCENTRIC Reconstruction for fast high-resolution neurometabolic imaging
• Cross-domain Denoising for Low-dose Multi-frame Spiral Computed Tomography
• Deep Unrolled Network for Video Super-Resolution
• Adaptive Local Neighborhood-based Neural Networks for MR Image Reconstruction from Undersampled Data
• MRI Recovery with Self-Calibrated Denoisers without Fully-Sampled Data
• Controlled Learning of Pointwise Nonlinearities in Neural-Network-Like Architectures
• TAEN: A Model-Constrained Tikhonov Autoencoder Network for Forward and Inverse Problems
• An unsupervised method for MRI recovery: Deep image prior with structured sparsity
• Magnitude-image based data-consistent deep learning method for MRI super resolution
• A Statistical Framework to Investigate the Optimality of Signal-Reconstruction Methods
• Robust Simultaneous Multislice MRI Reconstruction Using Slice-Wise Learned Generative Diffusion Priors
• Swap-Net: A Memory-Efficient 2.5D Network for Sparse-View 3D Cone Beam CT Reconstruction
• Generalized Deep Learning-based Proximal Gradient Descent for MR Reconstruction
• Cine cardiac MRI reconstruction using a convolutional recurrent network with refinement
• Stability Bounds for the Unfolded Forward-Backward Algorithm
• Patient-Adaptive and Learned MRI Data Undersampling Using Neighborhood Clustering
• Product-of-Gaussian-Mixture Diffusion Models for Joint Nonlinear MRI Reconstruction