Calibrate, Emulate, Sample

Abstract

Many parameter estimation problems arising in applications are best cast in the framework of Bayesian inversion. This allows not only for an estimate of the parameters, but also for the quantification of uncertainties in the estimates. Often in such problems the parameter-to-data map is very expensive to evaluate, and computing derivatives of the map, or derivative-adjoints, may not be feasible. Additionally, in many applications only noisy evaluations of the map may be available. We propose an approach to Bayesian inversion in such settings that builds on the derivative-free optimization capabilities of ensemble Kalman inversion methods. The overarching approach is to first use ensemble Kalman sampling (EKS) to calibrate the unknown parameters to fit the data; second, to use the output of the EKS to emulate the parameter-to-data map; third, to sample from an approximate Bayesian posterior distribution in which the parameter-to-data map is replaced by its emulator. This results in a principled approach to approximate Bayesian inference that requires only a small number of evaluations of the (possibly noisy approximation of the) parameter-to-data map. It does not require derivatives of this map, but instead leverages the documented power of ensemble Kalman methods. Furthermore, the EKS has the desirable property that it evolves the parameter ensembles towards the regions in which the bulk of the parameter posterior mass is located, thereby locating them well for the emulation phase of the methodology. In essence, the EKS methodology provides a cheap solution to the design problem of where to place points in parameter space to efficiently train an emulator of the parameter-to-data map for the purposes of Bayesian inversion.

References in corpus (17)

• General state space Markov chains and MCMC algorithms
• The prior can generally only be understood in the context of the likelihood
• The Ensemble Kalman Filter for Inverse Problems
• Earth System Modeling 2.0: A Blueprint for Models That Learn From Observations and Targeted High-Resolution Simulations
• Data Assimilation: A Mathematical Introduction
• Ensemble Kalman Inversion: A Derivative-Free Technique For Machine Learning Tasks
• Adaptive multi-fidelity polynomial chaos approach to Bayesian inference in inverse problems
• Adaptive construction of surrogates for the Bayesian solution of inverse problems
• Calibration and Uncertainty Quantification of Convective Parameters in an Idealized GCM
• Accelerating Asymptotically Exact MCMC for Computationally Intensive Models via Local Approximations
• Ensemble Kalman Methods With Constraints
• Emulation of Higher-Order Tensors in Manifold Monte Carlo Methods for Bayesian Inverse Problems
• Structured Bayesian Gaussian process latent variable model: applications to data-driven dimensionality reduction and high-dimensional inversion
• Note on Interacting Langevin Diffusions: Gradient Structure and Ensemble Kalman Sampler by Garbuno-Inigo, Hoffmann, Li and Stuart
• An adaptive surrogate modeling based on deep neural networks for large-scale Bayesian inverse problems
• Multilevel adaptive sparse Leja approximations for Bayesian inverse problems
• Inverse Problems and Data Assimilation

Cited by in corpus (20)

• Bridging observation, theory and numerical simulation of the ocean using Machine Learning
• Calibration and Uncertainty Quantification of Convective Parameters in an Idealized GCM
• Are General Circulation Models obsolete?
• Model calibration using ESEm v1.0.0 -- an open, scalable Earth System Emulator
• Machine learning for weather and climate are worlds apart
• Combining machine learning and data assimilation to forecast dynamical systems from noisy partial observations
• Parameter uncertainty quantification in an idealized GCM with a seasonal cycle
• Uncertainty Quantification and Propagation in Surrogate-based Bayesian Inference
• Explainable Offline-Online Training of Neural Networks for Parameterizations: A 1D Gravity Wave-QBO Testbed in the Small-data Regime
• Localization in Ensemble Kalman inversion
• Adaptive Tikhonov strategies for stochastic ensemble Kalman inversion
• Semi-automatic tuning of coupled climate models with multiple intrinsic timescales: lessons learned from the Lorenz96 model
• Hyperparameter Optimization for Randomized Algorithms: A Case Study on Random Features
• A survey of Monte Carlo methods for noisy and costly densities with application to reinforcement learning and ABC
• Improve Unscented Kalman Inversion With Low-Rank Approximation and Reduced-Order Model
• Kernel-based parameter estimation of dynamical systems with unknown observation functions
• Nesterov Acceleration for Ensemble Kalman Inversion and Variants
• An efficient estimation of time-varying parameters of dynamic models by combining offline batch optimization and online data assimilation
• Ensemble-Based Experimental Design for Targeting Data Acquisition to Inform Climate Models
• Wind farm yaw control set-point optimization under model parameter uncertainty