Multi-Fidelity Bayesian Optimization via Deep Neural Networks

Abstract

Bayesian optimization (BO) is a popular framework to optimize black-box functions. In many applications, the objective function can be evaluated at multiple fidelities to enable a trade-off between the cost and accuracy. To reduce the optimization cost, many multi-fidelity BO methods have been proposed. Despite their success, these methods either ignore or over-simplify the strong, complex correlations across the fidelities, and hence can be inefficient in estimating the objective function. To address this issue, we propose Deep Neural Network Multi-Fidelity Bayesian Optimization (DNN-MFBO) that can flexibly capture all kinds of complicated relationships between the fidelities to improve the objective function estimation and hence the optimization performance. We use sequential, fidelity-wise Gauss-Hermite quadrature and moment-matching to fulfill a mutual information-based acquisition function, which is computationally tractable and efficient. We show the advantages of our method in both synthetic benchmark datasets and real-world applications in engineering design.

References in corpus (10)

• Adam: A Method for Stochastic Optimization
• Practical Bayesian Optimization of Machine Learning Algorithms
• Expectation Propagation for approximate Bayesian inference
• Scalable Bayesian Optimization Using Deep Neural Networks
• Predictive Entropy Search for Efficient Global Optimization of Black-box Functions
• Max-value Entropy Search for Efficient Bayesian Optimization
• Multi-fidelity Bayesian Optimisation with Continuous Approximations
• A General Framework for Multi-fidelity Bayesian Optimization with Gaussian Processes
• Multi-fidelity Bayesian Optimization with Max-value Entropy Search and its parallelization
• Practical Bayesian Optimization for Variable Cost Objectives

Cited by in corpus (7)

• A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
• A Latent Variable Approach for Non-Hierarchical Multi-Fidelity Adaptive Sampling
• Leveraging Trust for Joint Multi-Objective and Multi-Fidelity Optimization
• A Bayesian neural network approach to Multi-fidelity surrogate modelling
• Multi-fidelity Hierarchical Neural Processes
• Single- to multi-fidelity history-dependent learning with uncertainty quantification and disentanglement: application to data-driven constitutive modeling
• Graph Laplacian-based Bayesian Multi-fidelity Modeling