Bayesian Experimental Design for Model Discrepancy Calibration: A Rivalry between Kullback--Leibler Divergence and Wasserstein Distance
arXiv:2601.16425 · doi:10.1016/j.jcp.2025.114469
Abstract
Designing experiments that systematically gather data from complex physical systems is central to accelerating scientific discovery. While Bayesian experimental design (BED) provides a principled, information-based framework that integrates experimental planning with probabilistic inference, the selection of utility functions in BED is a long-standing and active topic, where different criteria emphasize different notions of information. Although Kullback--Leibler (KL) divergence has been one of the most common choices, recent studies have proposed Wasserstein distance as an alternative. In this work, we first employ a toy example to illustrate an issue of Wasserstein distance - the value of Wasserstein distance of a fixed-shape posterior depends on the relative position of its main mass within the support and can exhibit false rewards unrelated to information gain, especially with a non-informative prior (e.g., uniform distribution). We then further provide a systematic comparison between these two criteria through a classical source inversion problem in the BED literature, revealing that the KL divergence tends to lead to faster convergence in the absence of model discrepancy, while Wasserstein metrics provide more robust sequential BED results if model discrepancy is non-negligible. These findings clarify the trade-offs between KL divergence and Wasserstein metrics for the utility function and provide guidelines for selecting suitable criteria in practical BED applications.
References in corpus (19)
- Variational Inference: A Review for Statisticians
- DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators
- Machine Learning for Fluid Mechanics
- Turbulence Modeling in the Age of Data
- A Physics Informed Machine Learning Approach for Reconstructing Reynolds Stress Modeling Discrepancies Based on DNS Data
- A Comprehensive Physics-Informed Machine Learning Framework for Predictive Turbulence Modeling
- Physics-Informed Machine Learning Approach for Augmenting Turbulence Models: A Comprehensive Framework
- The Ensemble Kalman Filter for Inverse Problems
- Gradient-based stochastic optimization methods in Bayesian experimental design
- Fast Bayesian experimental design: Laplace-based importance sampling for the expected information gain
- The Ensemble Kalman Filter: A Signal Processing Perspective
- Optimal experimental design: Formulations and computations
- Ensemble Kalman Inversion: mean-field limit and convergence analysis
- Bayesian Sequential Optimal Experimental Design for Nonlinear Models Using Policy Gradient Reinforcement Learning
- Unbiased MLMC stochastic gradient-based optimization of Bayesian experimental designs
- Neural Dynamical Operator: Continuous Spatial-Temporal Model with Gradient-Based and Derivative-Free Optimization Methods
- Variational Bayesian Optimal Experimental Design with Normalizing Flows
- CGNSDE: Conditional Gaussian Neural Stochastic Differential Equation for Modeling Complex Systems and Data Assimilation
- Active Learning of Model Discrepancy with Bayesian Experimental Design