BOSH: Bayesian Optimization by Sampling Hierarchically
arXiv:2007.00939
Abstract
Deployments of Bayesian Optimization (BO) for functions with stochastic evaluations, such as parameter tuning via cross validation and simulation optimization, typically optimize an average of a fixed set of noisy realizations of the objective function. However, disregarding the true objective function in this manner finds a high-precision optimum of the wrong function. To solve this problem, we propose Bayesian Optimization by Sampling Hierarchically (BOSH), a novel BO routine pairing a hierarchical Gaussian process with an information-theoretic framework to generate a growing pool of realizations as the optimization progresses. We demonstrate that BOSH provides more efficient and higher-precision optimization than standard BO across synthetic benchmarks, simulation optimization, reinforcement learning and hyper-parameter tuning tasks.
References in corpus (11)
- Practical Bayesian Optimization of Machine Learning Algorithms
- Determinantal point processes for machine learning
- Predictive Entropy Search for Efficient Global Optimization of Black-box Functions
- Bayesian optimization for materials design
- Batch Bayesian Optimization via Local Penalization
- Max-value Entropy Search for Efficient Bayesian Optimization
- Multi-fidelity Bayesian Optimisation with Continuous Approximations
- Emulation of physical processes with Emukit
- Multi-fidelity Bayesian Optimization with Max-value Entropy Search and its parallelization
- Using J-K fold Cross Validation to Reduce Variance When Tuning NLP Models
- Practical Bayesian Optimization for Variable Cost Objectives