High-Dimensional Bayesian Optimization via Additive Models with Overlapping Groups

Abstract

Bayesian optimization (BO) is a popular technique for sequential black-box function optimization, with applications including parameter tuning, robotics, environmental monitoring, and more. One of the most important challenges in BO is the development of algorithms that scale to high dimensions, which remains a key open problem despite recent progress. In this paper, we consider the approach of Kandasamy et al. (2015), in which the high-dimensional function decomposes as a sum of lower-dimensional functions on subsets of the underlying variables. In particular, we significantly generalize this approach by lifting the assumption that the subsets are disjoint, and consider additive models with arbitrary overlap among the subsets. By representing the dependencies via a graph, we deduce an efficient message passing algorithm for optimizing the acquisition function. In addition, we provide an algorithm for learning the graph from samples based on Gibbs sampling. We empirically demonstrate the effectiveness of our methods on both synthetic and real-world data.

Cited by in corpus (14)

• Tuning Hyperparameters without Grad Students: Scalable and Robust Bayesian Optimisation with Dragonfly
• Re-Examining Linear Embeddings for High-Dimensional Bayesian Optimization
• Learning Search Space Partition for Black-box Optimization using Monte Carlo Tree Search
• Scalable Constrained Bayesian Optimization
• Think Global and Act Local: Bayesian Optimisation over High-Dimensional Categorical and Mixed Search Spaces
• Additive Tree-Structured Conditional Parameter Spaces in Bayesian Optimization: A Novel Covariance Function and a Fast Implementation
• Corruption-Tolerant Gaussian Process Bandit Optimization
• Trading Convergence Rate with Computational Budget in High Dimensional Bayesian Optimization
• Learning to Guide Random Search
• Additive Tree-Structured Covariance Function for Conditional Parameter Spaces in Bayesian Optimization
• Gaussian Process Bandit Optimization with Few Batches
• Parameter Optimization using high-dimensional Bayesian Optimization
• Adversarial Attacks on Gaussian Process Bandits
• Scaling Bayesian Optimization With Game Theory