Bayesian Optimisation over Multiple Continuous and Categorical Inputs

Abstract

Efficient optimisation of black-box problems that comprise both continuous and categorical inputs is important, yet poses significant challenges. We propose a new approach, Continuous and Categorical Bayesian Optimisation (CoCaBO), which combines the strengths of multi-armed bandits and Bayesian optimisation to select values for both categorical and continuous inputs. We model this mixed-type space using a Gaussian Process kernel, designed to allow sharing of information across multiple categorical variables, each with multiple possible values; this allows CoCaBO to leverage all available data efficiently. We extend our method to the batch setting and propose an efficient selection procedure that dynamically balances exploration and exploitation whilst encouraging batch diversity. We demonstrate empirically that our method outperforms existing approaches on both synthetic and real-world optimisation tasks with continuous and categorical inputs.

16 pages

References in corpus (11)

• Scikit-learn: Machine Learning in Python
• Practical Bayesian Optimization of Machine Learning Algorithms
• A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning
• A Tutorial on Bayesian Optimization
• NAS-Bench-101: Towards Reproducible Neural Architecture Search
• Batch Bayesian Optimization via Local Penalization
• High Dimensional Bayesian Optimisation and Bandits via Additive Models
• The Parallel Knowledge Gradient Method for Batch Bayesian Optimization
• Predictive Entropy Search for Bayesian Optimization with Unknown Constraints
• Fast Information-theoretic Bayesian Optimisation
• MANAS: Multi-Agent Neural Architecture Search

Cited by in corpus (5)

• Gryffin: An algorithm for Bayesian optimization of categorical variables informed by expert knowledge
• Are we Forgetting about Compositional Optimisers in Bayesian Optimisation?
• Hybrid Parameter Search and Dynamic Model Selection for Mixed-Variable Bayesian Optimization
• Kernels over Sets of Finite Sets using RKHS Embeddings, with Application to Bayesian (Combinatorial) Optimization
• Black-box Mixed-Variable Optimisation using a Surrogate Model that Satisfies Integer Constraints