Practical heteroskedastic Gaussian process modeling for large simulation experiments

Abstract

We present a unified view of likelihood based Gaussian progress regression for simulation experiments exhibiting input-dependent noise. Replication plays an important role in that context, however previous methods leveraging replicates have either ignored the computational savings that come from such design, or have short-cut full likelihood-based inference to remain tractable. Starting with homoskedastic processes, we show how multiple applications of a well-known Woodbury identity facilitate inference for all parameters under the likelihood (without approximation), bypassing the typical full-data sized calculations. We then borrow a latent-variable idea from machine learning to address heteroskedasticity, adapting it to work within the same thrifty inferential framework, thereby simultaneously leveraging the computational and statistical efficiency of designs with replication. The result is an inferential scheme that can be characterized as single objective function, complete with closed form derivatives, for rapid library-based optimization. Illustrations are provided, including real-world simulation experiments from manufacturing and the management of epidemics.

33 pages, 7 figures

References in corpus (5)

• Composite Gaussian process models for emulating expensive functions
• Input Warping for Bayesian Optimization of Non-stationary Functions
• Accurate emulators for large-scale computer experiments
• Sequential Design for Ranking Response Surfaces
• Single Nugget Kriging

Cited by in corpus (34)

• A review of predictive uncertainty estimation with machine learning
• Replication or exploration? Sequential design for stochastic simulation experiments
• Global sensitivity analysis for stochastic simulators based on generalized lambda surrogate models
• Emulation of stochastic simulators using generalized lambda models
• Stochastic polynomial chaos expansions to emulate stochastic simulators
• A spectral surrogate model for stochastic simulators computed from trajectory samples
• VECMAtk: A Scalable Verification, Validation and Uncertainty Quantification Toolkit for Scientific Simulations
• Adaptive Design of Experiments for Conservative Estimation of Excursion Sets
• Task-Adaptive Robot Learning from Demonstration with Gaussian Process Models under Replication
• A Large Scale Spatio-temporal Binomial Regression Model for Estimating Seroprevalence Trends
• Sequential design of multi-fidelity computer experiments: maximizing the rate of stepwise uncertainty reduction
• Assessing Bayes factor surfaces using interactive visualization and computer surrogate modeling
• Evaluating Gaussian Process Metamodels and Sequential Designs for Noisy Level Set Estimation
• Achieving Robustness to Aleatoric Uncertainty with Heteroscedastic Bayesian Optimisation
• Large-scale Heteroscedastic Regression via Gaussian Process
• Reliability analysis for non-deterministic limit-states using stochastic emulators
• Sparse Polynomial Chaos Expansion for Universal Stochastic Kriging
• Sensitivity Analysis in the Presence of Intrinsic Stochasticity for Discrete Fracture Network Simulations
• Future Proofing a Building Design Using History Matching Inspired Level-Set Techniques
• Predicting the Output From a Stochastic Computer Model When a Deterministic Approximation is Available
• Parameter and Uncertainty Estimation for Dynamical Systems Using Surrogate Stochastic Processes
• A portfolio approach to massively parallel Bayesian optimization
• mlOSP: Towards a Unified Implementation of Regression Monte Carlo Algorithms
• Distance-distributed design for Gaussian process surrogates
• Model bias identification for Bayesian calibration of stochastic digital twins of bridges
• Goal-oriented adaptive sampling under random field modelling of response probability distributions
• Controlling Sources of Inaccuracy in Stochastic Kriging
• Accuracy-Reliability Cost Function for Empirical Variance Estimation
• Diagnostics for Stochastic Gaussian Process Emulators
• The Kalai-Smorodinski solution for many-objective Bayesian optimization
• Learning Arbitrary Quantities of Interest from Expensive Black-Box Functions through Bayesian Sequential Optimal Design
• On-site surrogates for large-scale calibration
• Scaling Bayesian Optimization With Game Theory
• Achieving Diversity in Objective Space for Sample-efficient Search of Multiobjective Optimization Problems