Dealing with Integer-valued Variables in Bayesian Optimization with Gaussian Processes

Abstract

Bayesian optimization (BO) methods are useful for optimizing functions that are expensive to evaluate, lack an analytical expression and whose evaluations can be contaminated by noise. These methods rely on a probabilistic model of the objective function, typically a Gaussian process (GP), upon which an acquisition function is built. This function guides the optimization process and measures the expected utility of performing an evaluation of the objective at a new point. GPs assume continous input variables. When this is not the case, such as when some of the input variables take integer values, one has to introduce extra approximations. A common approach is to round the suggested variable value to the closest integer before doing the evaluation of the objective. We show that this can lead to problems in the optimization process and describe a more principled approach to account for input variables that are integer-valued. We illustrate in both synthetic and a real experiments the utility of our approach, which significantly improves the results of standard BO methods on problems involving integer-valued variables.

7 pages

References in corpus (4)

• Practical Bayesian Optimization of Machine Learning Algorithms
• Scalable Bayesian Optimization Using Deep Neural Networks
• Predictive Entropy Search for Efficient Global Optimization of Black-box Functions
• Predictive Entropy Search for Multi-objective Bayesian Optimization with Constraints

Cited by in corpus (37)

• Hyper-Parameter Optimization: A Review of Algorithms and Applications
• Gryffin: An algorithm for Bayesian optimization of categorical variables informed by expert knowledge
• SMT 2.0: A Surrogate Modeling Toolbox with a focus on Hierarchical and Mixed Variables Gaussian Processes
• SpArSe: Sparse Architecture Search for CNNs on Resource-Constrained Microcontrollers
• An Artificial Intelligence (AI) workflow for catalyst design and optimization
• Golem: An algorithm for robust experiment and process optimization
• Bayesian Optimization For Multi-Objective Mixed-Variable Problems
• A mixed-categorical correlation kernel for Gaussian process
• Multi-fidelity Bayesian Optimization: A Review
• Suggesting Cooking Recipes Through Simulation and Bayesian Optimization
• RIBBON: Cost-Effective and QoS-Aware Deep Learning Model Inference using a Diverse Pool of Cloud Computing Instances
• Are we Forgetting about Compositional Optimisers in Bayesian Optimisation?
• Hyperparameter Optimization via Sequential Uniform Designs
• Hybrid Parameter Search and Dynamic Model Selection for Mixed-Variable Bayesian Optimization
• System Architecture Optimization Strategies: Dealing with Expensive Hierarchical Problems
• Bayesian optimization for mixed variables using an adaptive dimension reduction process: applications to aircraft design
• Hyper-parameter estimation method with particle swarm optimization
• Continuous surrogate-based optimization algorithms are well-suited for expensive discrete problems
• Surrogate-based optimization of system architectures subject to hidden constraints
• Kernels over Sets of Finite Sets using RKHS Embeddings, with Application to Bayesian (Combinatorial) Optimization
• High-dimensional mixed-categorical Gaussian processes with application to multidisciplinary design optimization for a green aircraft
• Combining Latent Space and Structured Kernels for Bayesian Optimization over Combinatorial Spaces
• Bayesian Optimization over Hybrid Spaces
• Towards Automatic Bayesian Optimization: A first step involving acquisition functions
• Black-box Mixed-Variable Optimisation using a Surrogate Model that Satisfies Integer Constraints
• Non-smooth Bayesian Optimization in Tuning Problems
• Multi-Output Gaussian Processes for Multi-Population Longevity Modeling
• A distance for mixed-variable and hierarchical domains with meta variables
• Bayesian Variational Optimization for Combinatorial Spaces
• BORE: Bayesian Optimization by Density-Ratio Estimation
• Bayesian Optimization for Categorical and Category-Specific Continuous Inputs
• Batch Multi-Fidelity Bayesian Optimization with Deep Auto-Regressive Networks
• Group Heterogeneity Assessment for Multilevel Models
• Data-driven parameterization refinement for the structural optimization of cruise ship hulls
• CLEAR: Cue Learning using Evolution for Accurate Recognition Applied to Sustainability Data Extraction
• Explaining Inference Queries with Bayesian Optimization
• Hyper-optimization with Gaussian Process and Differential Evolution Algorithm