Deep Gaussian Processes for Multi-fidelity Modeling

Abstract

Multi-fidelity methods are prominently used when cheaply-obtained, but possibly biased and noisy, observations must be effectively combined with limited or expensive true data in order to construct reliable models. This arises in both fundamental machine learning procedures such as Bayesian optimization, as well as more practical science and engineering applications. In this paper we develop a novel multi-fidelity model which treats layers of a deep Gaussian process as fidelity levels, and uses a variational inference scheme to propagate uncertainty across them. This allows for capturing nonlinear correlations between fidelities with lower risk of overfitting than existing methods exploiting compositional structure, which are conversely burdened by structural assumptions and constraints. We show that the proposed approach makes substantial improvements in quantifying and propagating uncertainty in multi-fidelity set-ups, which in turn improves their effectiveness in decision making pipelines.

References in corpus (7)

• Auto-Encoding Variational Bayes
• Gaussian Processes for Big Data
• Random Feature Expansions for Deep Gaussian Processes
• Avoiding pathologies in very deep networks
• Deep Multi-fidelity Gaussian Processes
• Large Scale Variable Fidelity Surrogate Modeling
• The Multi-fidelity Multi-armed Bandit

Cited by in corpus (14)

• Multi-fidelity regression using artificial neural networks: efficient approximation of parameter-dependent output quantities
• Multi-fidelity modeling with different input domain definitions using Deep Gaussian Processes
• A Latent Variable Approach for Non-Hierarchical Multi-Fidelity Adaptive Sampling
• Model-based Asynchronous Hyperparameter and Neural Architecture Search
• Multi-resolution Multi-task Gaussian Processes
• GIBBON: General-purpose Information-Based Bayesian OptimisatioN
• Overview of Gaussian process based multi-fidelity techniques with variable relationship between fidelities
• Multi-Fidelity High-Order Gaussian Processes for Physical Simulation
• GAN-MDF: A Method for Multi-fidelity Data Fusion in Digital Twins
• Batch Multi-Fidelity Bayesian Optimization with Deep Auto-Regressive Networks
• Deep Coregionalization for the Emulation of Spatial-Temporal Fields
• Conditional Deep Gaussian Processes: multi-fidelity kernel learning
• Gemini: Dynamic Bias Correction for Autonomous Experimentation and Molecular Simulation
• On MCMC for variationally sparse Gaussian processes: A pseudo-marginal approach