Bayesian Optimization over Hybrid Spaces

Abstract

We consider the problem of optimizing hybrid structures (mixture of discrete and continuous input variables) via expensive black-box function evaluations. This problem arises in many real-world applications. For example, in materials design optimization via lab experiments, discrete and continuous variables correspond to the presence/absence of primitive elements and their relative concentrations respectively. The key challenge is to accurately model the complex interactions between discrete and continuous variables. In this paper, we propose a novel approach referred as Hybrid Bayesian Optimization (HyBO) by utilizing diffusion kernels, which are naturally defined over continuous and discrete variables. We develop a principled approach for constructing diffusion kernels over hybrid spaces by utilizing the additive kernel formulation, which allows additive interactions of all orders in a tractable manner. We theoretically analyze the modeling strength of additive hybrid kernels and prove that it has the universal approximation property. Our experiments on synthetic and six diverse real-world benchmarks show that HyBO significantly outperforms the state-of-the-art methods.

14 pages, 18 figures

References in corpus (5)

• An Easy-to-use Real-world Multi-objective Optimization Problem Suite
• An Open Source AutoML Benchmark
• Max-value Entropy Search for Multi-Objective Bayesian Optimization with Constraints
• Learning with Hierarchical Gaussian Kernels
• Scalable Combinatorial Bayesian Optimization with Tractable Statistical models

Cited by in corpus (1)

• Hybrid Parameter Search and Dynamic Model Selection for Mixed-Variable Bayesian Optimization