Multi-fidelity Bayesian Optimization: A Review

Abstract

Resided at the intersection of multi-fidelity optimization (MFO) and Bayesian optimization (BO), MF BO has found a niche in solving expensive engineering design optimization problems, thanks to its advantages in incorporating physical and mathematical understandings of the problems, saving resources, addressing exploitation-exploration trade-off, considering uncertainty, and processing parallel computing. The increasing number of works dedicated to MF BO suggests the need for a comprehensive review of this advanced optimization technique. In this paper, we survey recent developments of two essential ingredients of MF BO: Gaussian process (GP) based MF surrogates and acquisition functions. We first categorize the existing MF modeling methods and MFO strategies to locate MF BO in a large family of surrogate-based optimization and MFO algorithms. We then exploit the common properties shared between the methods from each ingredient of MF BO to describe important GP-based MF surrogate models and review various acquisition functions. By doing so, we expect to provide a structured understanding of MF BO. Finally, we attempt to reveal important aspects that require further research for applications of MF BO in solving intricate yet important design optimization problems, including constrained optimization, high-dimensional optimization, optimization under uncertainty, and multi-objective optimization.

References in corpus (18)

• Automatic chemical design using a data-driven continuous representation of molecules
• A composite neural network that learns from multi-fidelity data: Application to function approximation and inverse PDE problems
• Active subspace methods in theory and practice: applications to kriging surfaces
• Derivative-free optimization methods
• Machine Learning in Aerodynamic Shape Optimization
• Dealing with Integer-valued Variables in Bayesian Optimization with Gaussian Processes
• Dealing with Categorical and Integer-valued Variables in Bayesian Optimization with Gaussian Processes
• Review of multi-fidelity models
• Adaptive multi-fidelity polynomial chaos approach to Bayesian inference in inverse problems
• Greed is Good: Exploration and Exploitation Trade-offs in Bayesian Optimisation
• Machine learning materials physics: Surrogate optimization and multi-fidelity algorithms predict precipitate morphology in an alternative to phase field dynamics
• Bayesian Optimization with Output-Weighted Optimal Sampling
• Upper Trust Bound Feasibility Criterion for Mixed Constrained Bayesian Optimization with Application to Aircraft Design
• Multi-Objective Bayesian Optimization for Accelerator Tuning
• Multi-fidelity modeling with different input domain definitions using Deep Gaussian Processes
• mfEGRA: Multifidelity Efficient Global Reliability Analysis through Active Learning for Failure Boundary Location
• Global sensitivity analysis for optimization with variable selection
• Adaptive Reliability Analysis for Multi-fidelity Models using a Collective Learning Strategy