Practical Bayesian Optimization of Objectives with Conditioning Variables
arXiv:2002.09996
Abstract
Bayesian optimization is a class of data efficient model based algorithms typically focused on global optimization. We consider the more general case where a user is faced with multiple problems that each need to be optimized conditional on a state variable, for example given a range of cities with different patient distributions, we optimize the ambulance locations conditioned on patient distribution. Given partitions of CIFAR-10, we optimize CNN hyperparameters for each partition. Similarity across objectives boosts optimization of each objective in two ways: in modelling by data sharing across objectives, and also in acquisition by quantifying how a single point on one objective can provide benefit to all objectives. For this we propose a framework for conditional optimization: ConBO. This can be built on top of a range of acquisition functions and we propose a new Hybrid Knowledge Gradient acquisition function. The resulting method is intuitive and theoretically grounded, performs either similar to or significantly better than recently published works on a range of problems, and is easily parallelized to collect a batch of points.
arXiv comment22 pages
References in corpus (9)
- Adam: A Method for Stochastic Optimization
- Practical Bayesian Optimization of Machine Learning Algorithms
- A Tutorial on Bayesian Optimization
- Batch Bayesian Optimization via Local Penalization
- Max-value Entropy Search for Efficient Bayesian Optimization
- Scalable Global Optimization via Local Bayesian Optimization
- Parallel and Distributed Thompson Sampling for Large-scale Accelerated Exploration of Chemical Space
- Offline Contextual Bayesian Optimization for Nuclear Fusion
- Discretization-free Knowledge Gradient Methods for Bayesian Optimization