Variational Inference for Nonlinear Inverse Problems via Neural Net Kernels: Comparison to Bayesian Neural Networks, Application to Topology Optimization

Abstract

Inverse problems and, in particular, inferring unknown or latent parameters from data are ubiquitous in engineering simulations. A predominant viewpoint in identifying unknown parameters is Bayesian inference where both prior information about the parameters and the information from the observations via likelihood evaluations are incorporated into the inference process. In this paper, we adopt a similar viewpoint with a slightly different numerical procedure from standard inference approaches to provide insight about the localized behavior of unknown underlying parameters. We present a variational inference approach which mainly incorporates the observation data in a point-wise manner, i.e. we invert a limited number of observation data leveraging the gradient information of the forward map with respect to parameters, and find true individual samples of the latent parameters when the forward map is noise-free and one-to-one. For statistical calculations (as the ultimate goal in simulations), a large number of samples are generated from a trained neural network which serves as a transport map from the prior to posterior latent parameters. Our neural network machinery, developed as part of the inference framework and referred to as Neural Net Kernels (NNK), is based on hierarchical (deep) kernels which provide greater flexibility for training compared to standard neural networks. We showcase the effectiveness of our inference procedure in identifying bimodal and irregular distributions compared to a number of approaches including Markov Chain Monte Carlo sampling approaches and a Bayesian neural network approach.

References in corpus (16)

• Bayesian Inference with Optimal Maps
• Data-Driven Model Reduction for the Bayesian Solution of Inverse Problems
• Geometric MCMC for Infinite-Dimensional Inverse Problems
• Sparse Deterministic Approximation of Bayesian Inverse Problems
• Kernel Flows: from learning kernels from data into the abyss
• The statistical finite element method (statFEM) for coherent synthesis of observation data and model predictions
• Neural-net-induced Gaussian process regression for function approximation and PDE solution
• Accelerating MCMC with active subspaces
• Solving Large-Scale PDE-constrained Bayesian Inverse Problems with Riemann Manifold Hamiltonian Monte Carlo
• Numerical Integration in Multiple Dimensions with Designed Quadrature
• Emulation of Higher-Order Tensors in Manifold Monte Carlo Methods for Bayesian Inverse Problems
• Hessian-based adaptive sparse quadrature for infinite-dimensional Bayesian inverse problems
• Metropolized Randomized Maximum Likelihood for sampling from multimodal distributions
• Parametric Topology Optimization with Multi-Resolution Finite Element Models
• Bayesian learning of orthogonal embeddings for multi-fidelity Gaussian Processes
• Multilevel adaptive sparse Leja approximations for Bayesian inverse problems