MCMC-Net: Accelerating Markov Chain Monte Carlo with Neural Networks for Inverse Problems

Abstract

In many computational problems, using the Markov Chain Monte Carlo (MCMC) can be prohibitively time-consuming. We propose MCMC-Net, a simple yet efficient way to accelerate MCMC via neural networks. The key idea of our approach is to substitute the true likelihood function of the MCMC method with a neural operator based surrogate. We extensively evaluate the accuracy and speedup of our method on three different PDE-based inverse problems where likelihood computations are computationally expensive, namely electrical impedance tomography, diffuse optical tomography, and quantitative photoacoustic tomography. MCMC-Net performs similar to the classical likelihood counterpart but with a significant speedup. We conjecture that the method can be applied to any problem with a sufficiently expensive likelihood function. We also analyze MCMC-Net in a theoretical setting for the different use cases. We prove a universal approximation theorem-type result to show that the proposed network can approximate the mapping resulting from forward model evaluations to a desired accuracy. Furthermore, we establish convergence of the surrogate posterior to the true posterior under Hellinger distance.

arXiv comment41 pages, 14 figures, 11 tables

References in corpus (23)

• DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators
• Solving high-dimensional partial differential equations using deep learning
• General state space Markov chains and MCMC algorithms
• Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification
• Deep UQ: Learning deep neural network surrogate models for high dimensional uncertainty quantification
• FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operators
• DeepM&Mnet: Inferring the electroconvection multiphysics fields based on operator approximation by neural networks
• Multipole Graph Neural Operator for Parametric Partial Differential Equations
• Data-Driven Model Reduction for the Bayesian Solution of Inverse Problems
• Solving Inverse Problems With Deep Neural Networks -- Robustness Included?
• Optimal approximation of continuous functions by very deep ReLU networks
• Adaptive construction of surrogates for the Bayesian solution of inverse problems
• Scalable posterior approximations for large-scale Bayesian inverse problems via likelihood-informed parameter and state reduction
• Choose a Transformer: Fourier or Galerkin
• Learning Operators with Coupled Attention
• Convolutional Neural Operators for robust and accurate learning of PDEs
• Residual-based error correction for neural operator accelerated infinite-dimensional Bayesian inverse problems
• NOMAD: Nonlinear Manifold Decoders for Operator Learning
• Convergence Rates for Learning Linear Operators from Noisy Data
• Neural Inverse Operators for Solving PDE Inverse Problems
• Variable-Input Deep Operator Networks
• A deep surrogate approach to efficient Bayesian inversion in PDE and integral equation models
• Globally injective and bijective neural operators