SGD-Net: Efficient Model-Based Deep Learning with Theoretical Guarantees

Abstract

Deep unfolding networks have recently gained popularity in the context of solving imaging inverse problems. However, the computational and memory complexity of data-consistency layers within traditional deep unfolding networks scales with the number of measurements, limiting their applicability to large-scale imaging inverse problems. We propose SGD-Net as a new methodology for improving the efficiency of deep unfolding through stochastic approximations of the data-consistency layers. Our theoretical analysis shows that SGD-Net can be trained to approximate batch deep unfolding networks to an arbitrary precision. Our numerical results on intensity diffraction tomography and sparse-view computed tomography show that SGD-Net can match the performance of the batch network at a fraction of training and testing complexity.

References in corpus (30)

• Adam: A Method for Stochastic Optimization
• Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising
• Deep Convolutional Neural Network for Inverse Problems in Imaging
• Low-Dose CT with a Residual Encoder-Decoder Convolutional Neural Network (RED-CNN)
• Fast Image Recovery Using Variable Splitting and Constrained Optimization
• MoDL: Model Based Deep Learning Architecture for Inverse Problems
• Learned Primal-dual Reconstruction
• A deep convolutional neural network using directional wavelets for low-dose X-ray CT reconstruction
• A Review of Convolutional Neural Networks for Inverse Problems in Imaging
• BM3D Frames and Variational Image Deblurring
• Solving ill-posed inverse problems using iterative deep neural networks
• CNN-Based Projected Gradient Descent for Consistent Image Reconstruction
• Non-convex Optimization for Machine Learning
• Model based learning for accelerated, limited-view 3D photoacoustic tomography
• Self-Supervised Learning of Physics-Guided Reconstruction Neural Networks without Fully-Sampled Reference Data
• Plug-and-Play Priors for Bright Field Electron Tomography and Sparse Interpolation
• Plug and play methods for magnetic resonance imaging (long version)
• An Online Plug-and-Play Algorithm for Regularized Image Reconstruction
• Hessian Schatten-Norm Regularization for Linear Inverse Problems
• RARE: Image Reconstruction using Deep Priors Learned without Ground Truth
• Efficient and accurate inversion of multiple scattering with deep learning
• High-throughput intensity diffraction tomography with a computational microscope
• J-MoDL: Joint Model-Based Deep Learning for Optimized Sampling and Reconstruction
• Plug-and-Play Methods Provably Converge with Properly Trained Denoisers
• Dense Recurrent Neural Networks for Accelerated MRI: History-Cognizant Unrolling of Optimization Algorithms
• Why gradient clipping accelerates training: A theoretical justification for adaptivity
• Learning optimal nonlinearities for iterative thresholding algorithms
• Provable Convergence of Plug-and-Play Priors with MMSE denoisers
• SIMBA: Scalable Inversion in Optical Tomography using Deep Denoising Priors
• Learning-based Image Reconstruction via Parallel Proximal Algorithm

Cited by in corpus (5)

• CoIL: Coordinate-based Internal Learning for Imaging Inverse Problems
• PtychoDV: Vision Transformer-Based Deep Unrolling Network for Ptychographic Image Reconstruction
• Learning Task-Specific Strategies for Accelerated MRI
• Pivotal Auto-Encoder via Self-Normalizing ReLU
• Swap-Net: A Memory-Efficient 2.5D Network for Sparse-View 3D Cone Beam CT Reconstruction