Sequential Bayesian optimal experimental design via approximate dynamic programming

Abstract

The design of multiple experiments is commonly undertaken via suboptimal strategies, such as batch (open-loop) design that omits feedback or greedy (myopic) design that does not account for future effects. This paper introduces new strategies for the optimal design of sequential experiments. First, we rigorously formulate the general sequential optimal experimental design (sOED) problem as a dynamic program. Batch and greedy designs are shown to result from special cases of this formulation. We then focus on sOED for parameter inference, adopting a Bayesian formulation with an information theoretic design objective. To make the problem tractable, we develop new numerical approaches for nonlinear design with continuous parameter, design, and observation spaces. We approximate the optimal policy by using backward induction with regression to construct and refine value function approximations in the dynamic program. The proposed algorithm iteratively generates trajectories via exploration and exploitation to improve approximation accuracy in frequently visited regions of the state space. Numerical results are verified against analytical solutions in a linear-Gaussian setting. Advantages over batch and greedy design are then demonstrated on a nonlinear source inversion problem where we seek an optimal policy for sequential sensing.

arXiv commentPreprint 34 pages, 12 figures (36 small figures). v1 submitted to the SIAM/ASA Journal on Uncertainty Quantification on April 27, 2016

References in corpus (4)

• Simulation-based optimal Bayesian experimental design for nonlinear systems
• Bayesian Inference with Optimal Maps
• Gradient-based stochastic optimization methods in Bayesian experimental design
• Rotating Stars and Revolving Planets: Bayesian Exploration of the Pulsating Sky

Cited by in corpus (12)

• Replication or exploration? Sequential design for stochastic simulation experiments
• Bayesian Optimization with Output-Weighted Optimal Sampling
• Bayesian Sequential Optimal Experimental Design for Nonlinear Models Using Policy Gradient Reinforcement Learning
• Sequential Bayesian optimal experimental design for structural reliability analysis
• Optimal Experimental Design Using A Consistent Bayesian Approach
• A fast and scalable computational framework for large-scale and high-dimensional Bayesian optimal experimental design
• Deep Adaptive Design: Amortizing Sequential Bayesian Experimental Design
• Design Analysis for Optimal Calibration of Diffusivity in Reactive Multilayers
• Variational Sequential Optimal Experimental Design using Reinforcement Learning
• Active Learning of Model Discrepancy with Bayesian Experimental Design
• Data-driven Sensor Placement for Predictive Applications: A Correlation-Assisted Attribution Framework (CAAF)
• Learning Arbitrary Quantities of Interest from Expensive Black-Box Functions through Bayesian Sequential Optimal Design