Faster Uncertainty Quantification for Inverse Problems with Conditional Normalizing Flows

Abstract

In inverse problems, we often have access to data consisting of paired samples $(x,y)\sim p_{X,Y}(x,y)$ where $y$ are partial observations of a physical system, and $x$ represents the unknowns of the problem. Under these circumstances, we can employ supervised training to learn a solution $x$ and its uncertainty from the observations $y$. We refer to this problem as the "supervised" case. However, the data $y\sim p_Y(y)$ collected at one point could be distributed differently than observations $y^{\prime }\sim p_Y^{\prime }(y^{\prime })$, relevant for a current set of problems. In the context of Bayesian inference, we propose a two-step scheme, which makes use of normalizing flows and joint data to train a conditional generator $q_{\theta }(x|y)$ to approximate the target posterior density $p_{X|Y}(x|y)$. Additionally, this preliminary phase provides a density function $q_{\theta }(x|y)$, which can be recast as a prior for the "unsupervised" problem, e.g.~when only the observations $y^{\prime }\sim p_Y^{\prime }(y^{\prime })$, a likelihood model $y^{\prime }|x$, and a prior on $x^{\prime }$ are known. We then train another invertible generator with output density $q_{\varphi }^{\prime }(x|y^{\prime })$ specifically for $y^{\prime }$, allowing us to sample from the posterior $p_{X|Y}^{\prime }(x|y^{\prime })$. We present some synthetic results that demonstrate considerable training speedup when reusing the pretrained network $q_{\theta }(x|y^{\prime })$ as a warm start or preconditioning for approximating $p_{X|Y}^{\prime }(x|y^{\prime })$, instead of learning from scratch. This training modality can be interpreted as an instance of transfer learning. This result is particularly relevant for large-scale inverse problems that employ expensive numerical simulations.

References in corpus (7)

• How transferable are features in deep neural networks?
• NICE: Non-linear Independent Components Estimation
• Deep Image Prior
• Density estimation using Real NVP
• Learning Likelihoods with Conditional Normalizing Flows
• Learned imaging with constraints and uncertainty quantification
• Fully reversible neural networks for large-scale surface and sub-surface characterization via remote sensing

Cited by in corpus (4)

• 3D Bayesian Variational Full Waveform Inversion
• An introduction to variational inference in Geophysical inverse problems
• Preconditioned training of normalizing flows for variational inference in inverse problems
• VIP -- Variational Inversion Package with example implementations of Bayesian tomographic imaging