Reconstructing unseen modalities and pathology with an efficient Recurrent Inference Machine

Abstract

Objective: To allow efficient learning using the Recurrent Inference Machine (RIM) for image reconstruction whereas not being strictly dependent on the training data distribution so that unseen modalities and pathologies are still accurately recovered. Methods: Theoretically, the RIM learns to solve the inverse problem of accelerated-MRI reconstruction whereas being robust to variable imaging conditions. The efficiency and generalization capabilities with different training datasets were studied, as well as recurrent network units with decreasing complexity: the Gated Recurrent Unit (GRU), the Minimal Gated Unit (MGU), and the Independently Recurrent Neural Network (IndRNN), to reduce inference times. Validation was performed against Compressed Sensing (CS) and further assessed based on data unseen during training. A pathology study was conducted by reconstructing simulated white matter lesions and prospectively undersampled data of a Multiple Sclerosis patient. Results: Training on a single modality of 3T $T_1$-weighted brain data appeared sufficient to also reconstruct 7T $T_2^{\ast }$-weighted brain and 3T $T_2$-weighted knee data. The IndRNN is an efficient recurrent unit, reducing inference time by 68\% compared to CS, whereas maintaining performance. The RIM was able to reconstruct lesions unseen during training more accurately than CS when trained on $T_2$-weighted knee data. Training on $T_1$-weighted brain data and on combined data slightly enhanced the signal compared to CS. Conclusion: The RIM is efficient when decreasing its complexity, which reduces the inference time, whereas still being able to reconstruct data and pathology that was unseen during training.

arXiv comment20 pages, 8 figures

References in corpus (16)

• A Survey on Deep Learning in Medical Image Analysis
• On the difficulty of training Recurrent Neural Networks
• Deep Convolutional Neural Network for Inverse Problems in Imaging
• Image reconstruction by domain transform manifold learning
• Compressed Sensing MRI Reconstruction using a Generative Adversarial Network with a Cyclic Loss
• Loss Functions for Neural Networks for Image Processing
• MR image reconstruction using deep density priors
• Deep Generative Adversarial Networks for Compressed Sensing Automates MRI
• Unrolled Optimization with Deep Priors
• Neural Proximal Gradient Descent for Compressive Imaging
• Recurrent Inference Machines for Solving Inverse Problems
• Unsupervised Deep Basis Pursuit: Learning inverse problems without ground-truth data
• $\Sigma$-net: Systematic Evaluation of Iterative Deep Neural Networks for Fast Parallel MR Image Reconstruction
• DuDoRNet: Learning a Dual-Domain Recurrent Network for Fast MRI Reconstruction with Deep T1 Prior
• Synergistic Reconstruction and Synthesis via Generative Adversarial Networks for Accelerated Multi-Contrast MRI
• i-RIM applied to the fastMRI challenge