PatchNR: Learning from Very Few Images by Patch Normalizing Flow Regularization

Abstract

Learning neural networks using only few available information is an important ongoing research topic with tremendous potential for applications. In this paper, we introduce a powerful regularizer for the variational modeling of inverse problems in imaging. Our regularizer, called patch normalizing flow regularizer (patchNR), involves a normalizing flow learned on small patches of very few images. In particular, the training is independent of the considered inverse problem such that the same regularizer can be applied for different forward operators acting on the same class of images. By investigating the distribution of patches versus those of the whole image class, we prove that our model is indeed a MAP approach. Numerical examples for low-dose and limited-angle computed tomography (CT) as well as superresolution of material images demonstrate that our method provides very high quality results. The training set consists of just six images for CT and one image for superresolution. Finally, we combine our patchNR with ideas from internal learning for performing superresolution of natural images directly from the low-resolution observation without knowledge of any high-resolution image.

References in corpus (31)

• Adam: A Method for Stochastic Optimization
• Auto-Encoding Variational Bayes
• Conditional Generative Adversarial Nets
• Deep Convolutional Neural Network for Inverse Problems in Imaging
• Variational Inference with Normalizing Flows
• Generative Modeling by Estimating Gradients of the Data Distribution
• Learned Primal-dual Reconstruction
• Plug-and-Play Priors for Bright Field Electron Tomography and Sparse Interpolation
• Guided Image Generation with Conditional Invertible Neural Networks
• Denoising Diffusion Restoration Models
• Computed Tomography Reconstruction Using Deep Image Prior and Learned Reconstruction Methods
• Solving Inverse Problems in Medical Imaging with Score-Based Generative Models
• The LoDoPaB-CT Dataset: A Benchmark Dataset for Low-Dose CT Reconstruction Methods
• Adversarial Regularizers in Inverse Problems
• Convolutional Proximal Neural Networks and Plug-and-Play Algorithms
• Coupling-based Invertible Neural Networks Are Universal Diffeomorphism Approximators
• Deep Unfolding with Normalizing Flow Priors for Inverse Problems
• Stabilizing Invertible Neural Networks Using Mixture Models
• Stochastic Normalizing Flows for Inverse Problems: a Markov Chains Viewpoint
• Multi-Resolution Data Fusion for Super Resolution Imaging
• Tails of Lipschitz Triangular Flows
• Wavelet Flow: Fast Training of High Resolution Normalizing Flows
• Gradient Step Denoiser for convergent Plug-and-Play
• Wasserstein Patch Prior for Image Superresolution
• The basins of attraction of the global minimizers of non-convex inverse problems with low-dimensional models in infinite dimension
• WPPNets and WPPFlows: The Power of Wasserstein Patch Priors for Superresolution
• End-to-end reconstruction meets data-driven regularization for inverse problems
• Regularising Inverse Problems with Generative Machine Learning Models
• CT Super Resolution via Zero Shot Learning
• Posterior Sampling for Image Restoration using Explicit Patch Priors
• CAFLOW: Conditional Autoregressive Flows

Cited by in corpus (3)

• Generalized Normalizing Flows via Markov Chains
• Inverse problem regularization with hierarchical variational autoencoders
• Fusion of JWST data: Demonstrating practical feasibility