An acceleration strategy for randomize-then-optimize sampling via deep neural networks

Abstract

Randomize-then-optimize (RTO) is widely used for sampling from posterior distributions in Bayesian inverse problems. However, RTO may be computationally intensive for complexity problems due to repetitive evaluations of the expensive forward model and its gradient. In this work, we present a novel strategy to substantially reduce the computation burden of RTO by using a goal-oriented deep neural networks (DNN) surrogate approach. In particular, the training points for the DNN-surrogate are drawn from a local approximated posterior distribution, and it is shown that the resulting algorithm can provide a flexible and efficient sampling algorithm, which converges to the direct RTO approach. We present a Bayesian inverse problem governed by a benchmark elliptic PDE to demonstrate the computational accuracy and efficiency of our new algorithm (i.e., DNN-RTO). It is shown that with our algorithm, one can significantly outperform the traditional RTO.

arXiv commentto appear in J. Comput. Math

References in corpus (11)

• Adam: A Method for Stochastic Optimization
• ADADELTA: An Adaptive Learning Rate Method
• Searching for Activation Functions
• Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification
• A composite neural network that learns from multi-fidelity data: Application to function approximation and inverse PDE problems
• Deep UQ: Learning deep neural network surrogate models for high dimensional uncertainty quantification
• Geometric MCMC for Infinite-Dimensional Inverse Problems
• Adaptive multi-fidelity polynomial chaos approach to Bayesian inference in inverse problems
• A random map implementation of implicit filters
• Emulation of Higher-Order Tensors in Manifold Monte Carlo Methods for Bayesian Inverse Problems
• Scalable optimization-based sampling on function space