Variational Bayesian Optimal Experimental Design with Normalizing Flows

Abstract

Bayesian optimal experimental design (OED) seeks experiments that maximize the expected information gain (EIG) in model parameters. Directly estimating the EIG using nested Monte Carlo is computationally expensive and requires an explicit likelihood. Variational OED (vOED), in contrast, estimates a lower bound of the EIG without likelihood evaluations by approximating the posterior distributions with variational forms, and then tightens the bound by optimizing its variational parameters. We introduce the use of normalizing flows (NFs) for representing variational distributions in vOED; we call this approach vOED-NFs. Specifically, we adopt NFs with a conditional invertible neural network architecture built from compositions of coupling layers, and enhanced with a summary network for data dimension reduction. We present Monte Carlo estimators to the lower bound along with gradient expressions to enable a gradient-based simultaneous optimization of the variational parameters and the design variables. The vOED-NFs algorithm is then validated in two benchmark problems, and demonstrated on a partial differential equation-governed application of cathodic electrophoretic deposition and an implicit likelihood case with stochastic modeling of aphid population. The findings suggest that a composition of 4--5 coupling layers is able to achieve lower EIG estimation bias, under a fixed budget of forward model runs, compared to previous approaches. The resulting NFs produce approximate posteriors that agree well with the true posteriors, able to capture non-Gaussian and multi-modal features effectively.

References in corpus (19)

• Representation Learning with Contrastive Predictive Coding
• NICE: Non-linear Independent Components Estimation
• Normalizing Flows: An Introduction and Review of Current Methods
• Estimating divergence functionals and the likelihood ratio by convex risk minimization
• Simulation-based optimal Bayesian experimental design for nonlinear systems
• MINE: Mutual Information Neural Estimation
• Bayesian Inference with Optimal Maps
• Gradient-based stochastic optimization methods in Bayesian experimental design
• Solving inverse problems using conditional invertible neural networks
• Fast Bayesian experimental design: Laplace-based importance sampling for the expected information gain
• Optimal experimental design: Formulations and computations
• Bayesian Sequential Optimal Experimental Design for Nonlinear Models Using Policy Gradient Reinforcement Learning
• On the representation and learning of monotone triangular transport maps
• Deep composition of tensor-trains using squared inverse Rosenblatt transports
• A layered multiple importance sampling scheme for focused optimal Bayesian experimental design
• Efficient Neural Network Approaches for Conditional Optimal Transport with Applications in Bayesian Inference
• Probabilistic Bayesian optimal experimental design using conditional normalizing flows
• On the Universality of Volume-Preserving and Coupling-Based Normalizing Flows
• Goal-Oriented Bayesian Optimal Experimental Design for Nonlinear Models using Markov Chain Monte Carlo

Cited by in corpus (5)

• Variational Sequential Optimal Experimental Design using Reinforcement Learning
• Optimal Experimental Design for Reliable Learning of History-Dependent Constitutive Laws
• Bayesian Experimental Design for Model Discrepancy Calibration: A Rivalry between Kullback--Leibler Divergence and Wasserstein Distance
• Bayesian Experimental Design for Model Discrepancy Calibration: An Auto-Differentiable Ensemble Kalman Inversion Approach
• Entropy-Informed Weighting Channel Normalizing Flow for Deep Generative Models