Think Global and Act Local: Bayesian Optimisation over High-Dimensional Categorical and Mixed Search Spaces
arXiv:2102.07188
Abstract
High-dimensional black-box optimisation remains an important yet notoriously challenging problem. Despite the success of Bayesian optimisation methods on continuous domains, domains that are categorical, or that mix continuous and categorical variables, remain challenging. We propose a novel solution -- we combine local optimisation with a tailored kernel design, effectively handling high-dimensional categorical and mixed search spaces, whilst retaining sample efficiency. We further derive convergence guarantee for the proposed approach. Finally, we demonstrate empirically that our method outperforms the current baselines on a variety of synthetic and real-world tasks in terms of performance, computational costs, or both.
arXiv commentICML 2021. 9 page, 6 figures (26 pages, 16 figures, 2 tables including references and appendices)
References in corpus (11)
- Practical Bayesian Optimization of Machine Learning Algorithms
- A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning
- GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration
- A Tutorial on Bayesian Optimization
- Scalable Global Optimization via Local Bayesian Optimization
- Batched Large-scale Bayesian Optimization in High-dimensional Spaces
- Bayesian Optimization of Combinatorial Structures
- Combinatorial Bayesian Optimization using the Graph Cartesian Product
- High-Dimensional Bayesian Optimization via Additive Models with Overlapping Groups
- Interpretable Neural Architecture Search via Bayesian Optimisation with Weisfeiler-Lehman Kernels
- Black-box Mixed-Variable Optimisation using a Surrogate Model that Satisfies Integer Constraints