Task adapted reconstruction for inverse problems

Abstract

The paper considers the problem of performing a task defined on a model parameter that is only observed indirectly through noisy data in an ill-posed inverse problem. A key aspect is to formalize the steps of reconstruction and task as appropriate estimators (non-randomized decision rules) in statistical estimation problems. The implementation makes use of (deep) neural networks to provide a differentiable parametrization of the family of estimators for both steps. These networks are combined and jointly trained against suitable supervised training data in order to minimize a joint differentiable loss function, resulting in an end-to-end task adapted reconstruction method. The suggested framework is generic, yet adaptable, with a plug-and-play structure for adjusting both the inverse problem and the task at hand. More precisely, the data model (forward operator and statistical model of the noise) associated with the inverse problem is exchangeable, e.g., by using neural network architecture given by a learned iterative method. Furthermore, any task that is encodable as a trainable neural network can be used. The approach is demonstrated on joint tomographic image reconstruction, classification and joint tomographic image reconstruction segmentation.

References in corpus (16)

• VoxelMorph: A Learning Framework for Deformable Medical Image Registration
• On Calibration of Modern Neural Networks
• Learned Primal-dual Reconstruction
• Solving ill-posed inverse problems using iterative deep neural networks
• 3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation
• Model based learning for accelerated, limited-view 3D photoacoustic tomography
• Deep Generative Adversarial Networks for Compressed Sensing Automates MRI
• Joint Image Reconstruction and Segmentation Using the Potts Model
• A Partially Learned Algorithm for Joint Photoacoustic Reconstruction and Segmentation
• Recurrent Generative Adversarial Networks for Proximal Learning and Automated Compressive Image Recovery
• Enhancing Compressed Sensing 4D Photoacoustic Tomography by Simultaneous Motion Estimation
• Indirect Image Registration with Large Diffeomorphic Deformations
• A New Variational Model for Joint Image Reconstruction and Motion Estimation in Spatiotemporal Imaging
• Learning to solve inverse problems using Wasserstein loss
• Simultaneous Reconstruction and Segmentation for Dynamic SPECT Imaging
• Spatiotemporal PET reconstruction using ML-EM with learned diffeomorphic deformation

Cited by in corpus (6)

• Deep Bayesian Inversion
• A study of why we need to reassess full reference image quality assessment with medical images
• Limited View Tomographic Reconstruction Using a Deep Recurrent Framework with Residual Dense Spatial-Channel Attention Network and Sinogram Consistency
• Multi-task MR Imaging with Iterative Teacher Forcing and Re-weighted Deep Learning
• Joint reconstruction-segmentation on graphs
• Regularized Compression of MRI Data: Modular Optimization of Joint Reconstruction and Coding