Recurrent Inference Machines for Solving Inverse Problems

Abstract

Much of the recent research on solving iterative inference problems focuses on moving away from hand-chosen inference algorithms and towards learned inference. In the latter, the inference process is unrolled in time and interpreted as a recurrent neural network (RNN) which allows for joint learning of model and inference parameters with back-propagation through time. In this framework, the RNN architecture is directly derived from a hand-chosen inference algorithm, effectively limiting its capabilities. We propose a learning framework, called Recurrent Inference Machines (RIM), in which we turn algorithm construction the other way round: Given data and a task, train an RNN to learn an inference algorithm. Because RNNs are Turing complete [1, 2] they are capable to implement any inference algorithm. The framework allows for an abstraction which removes the need for domain knowledge. We demonstrate in several image restoration experiments that this abstraction is effective, allowing us to achieve state-of-the-art performance on image denoising and super-resolution tasks and superior across-task generalization.

References in corpus (2)

• Auto-Encoding Variational Bayes
• On learning optimized reaction diffusion processes for effective image restoration

Cited by in corpus (43)

• Machine learning and the physical sciences
• The frontier of simulation-based inference
• Model based learning for accelerated, limited-view 3D photoacoustic tomography
• Results of the 2020 fastMRI Challenge for Machine Learning MR Image Reconstruction
• MR image reconstruction using deep density priors
• Deep Learning in Photoacoustic Tomography: Current approaches and future directions
• Learning to Optimize: A Primer and A Benchmark
• Deep Learning Methods for Solving Linear Inverse Problems: Research Directions and Paradigms
• Data-Driven Reconstruction of Gravitationally Lensed Galaxies using Recurrent Inference Machines
• RE-MIMO: Recurrent and Permutation Equivariant Neural MIMO Detection
• Deep Learning for Passive Synthetic Aperture Radar
• A Helmholtz equation solver using unsupervised learning: Application to transcranial ultrasound
• Probabilistic Autoencoder
• Learning to solve inverse problems using Wasserstein loss
• Analyzing interferometric observations of strong gravitational lenses with recurrent and convolutional neural networks
• Rotation Equivariant Proximal Operator for Deep Unfolding Methods in Image Restoration
• Meta-Learning Deep Energy-Based Memory Models
• Uncertainty Quantification with Generative Models
• Robust partial Fourier reconstruction for diffusion-weighted imaging using a recurrent convolutional neural network
• Assessment of Data Consistency through Cascades of Independently Recurrent Inference Machines for fast and robust accelerated MRI reconstruction
• i-RIM applied to the fastMRI challenge
• Deep Learning for Waveform Estimation and Imaging in Passive Radar
• Recurrent Localization Networks applied to the Lippmann-Schwinger Equation
• Optimization Algorithm Inspired Deep Neural Network Structure Design
• Hybrid Physical-Deep Learning Model for Astronomical Inverse Problems
• Variational Inference for Computational Imaging Inverse Problems
• Conditional Variational Autoencoder for Learned Image Reconstruction
• CosmicRIM : Reconstructing Early Universe by Combining Differentiable Simulations with Recurrent Inference Machines
• Quantifying Sources of Uncertainty in Deep Learning-Based Image Reconstruction
• Degrees of Freedom Analysis of Unrolled Neural Networks
• Deep MRI Reconstruction with Radial Subsampling
• Learning the Universe: Learning to Optimize Cosmic Initial Conditions with Non-Differentiable Structure Formation Models
• A proof-of-concept neural network for inferring parameters of a black hole from partial interferometric images of its shadow
• Quantifying Model Uncertainty in Inverse Problems via Bayesian Deep Gradient Descent
• Feasibility-based Fixed Point Networks
• Compressive-Sensing Data Reconstruction for Structural Health Monitoring: A Machine-Learning Approach
• Applications of Deep Learning for Ill-Posed Inverse Problems Within Optical Tomography
• R3L: Connecting Deep Reinforcement Learning to Recurrent Neural Networks for Image Denoising via Residual Recovery
• Equivariant neural networks for inverse problems
• Recurrent machines for likelihood-free inference
• Foreground color prediction through inverse compositing
• AAPM DL-Sparse-View CT Challenge Submission Report: Designing an Iterative Network for Fanbeam-CT with Unknown Geometry
• Reconstructing unseen modalities and pathology with an efficient Recurrent Inference Machine