A hierarchical expected improvement method for Bayesian optimization
arXiv:1911.07285
Abstract
The Expected Improvement (EI) method, proposed by Jones et al. (1998), is a widely-used Bayesian optimization method, which makes use of a fitted Gaussian process model for efficient black-box optimization. However, one key drawback of EI is that it is overly greedy in exploiting the fitted Gaussian process model for optimization, which results in suboptimal solutions even with large sample sizes. To address this, we propose a new hierarchical EI (HEI) framework, which makes use of a hierarchical Gaussian process model. HEI preserves a closed-form acquisition function, and corrects the over-greediness of EI by encouraging exploration of the optimization space. We then introduce hyperparameter estimation methods which allow HEI to mimic a fully Bayesian optimization procedure, while avoiding expensive Markov-chain Monte Carlo sampling steps. We prove the global convergence of HEI over a broad function space, and establish near-minimax convergence rates under certain prior specifications. Numerical experiments show the improvement of HEI over existing Bayesian optimization methods, for synthetic functions and a semiconductor manufacturing optimization problem.
References in corpus (6)
- Practical Bayesian Optimization of Machine Learning Algorithms
- A Bayesian approach to constrained single- and multi-objective optimization
- A novel class of stabilized greedy kernel approximation algorithms: Convergence, stability & uniform point distribution
- Variational Algorithms for Marginal MAP
- Convergence Guarantees for Gaussian Process Means With Misspecified Likelihoods and Smoothness
- Projected support points: a new method for high-dimensional data reduction