Combining Latent Space and Structured Kernels for Bayesian Optimization over Combinatorial Spaces

Abstract

We consider the problem of optimizing combinatorial spaces (e.g., sequences, trees, and graphs) using expensive black-box function evaluations. For example, optimizing molecules for drug design using physical lab experiments. Bayesian optimization (BO) is an efficient framework for solving such problems by intelligently selecting the inputs with high utility guided by a learned surrogate model. A recent BO approach for combinatorial spaces is through a reduction to BO over continuous spaces by learning a latent representation of structures using deep generative models (DGMs). The selected input from the continuous space is decoded into a discrete structure for performing function evaluation. However, the surrogate model over the latent space only uses the information learned by the DGM, which may not have the desired inductive bias to approximate the target black-box function. To overcome this drawback, this paper proposes a principled approach referred as LADDER. The key idea is to define a novel structure-coupled kernel that explicitly integrates the structural information from decoded structures with the learned latent space representation for better surrogate modeling. Our experiments on real-world benchmarks show that LADDER significantly improves over the BO over latent space method, and performs better or similar to state-of-the-art methods.

15 pages, 7 figures

References in corpus (7)

• Practical Bayesian Optimization of Machine Learning Algorithms
• BoTorch: A Framework for Efficient Monte-Carlo Bayesian Optimization
• Differentiable Expected Hypervolume Improvement for Parallel Multi-Objective Bayesian Optimization
• Gaussian Process Molecule Property Prediction with FlowMO
• Improving black-box optimization in VAE latent space using decoder uncertainty
• BOSS: Bayesian Optimization over String Spaces
• Scalable Combinatorial Bayesian Optimization with Tractable Statistical models