Cost-aware Multi-objective Bayesian optimisation

Abstract

The notion of expense in Bayesian optimisation generally refers to the uniformly expensive cost of function evaluations over the whole search space. However, in some scenarios, the cost of evaluation for black-box objective functions is non-uniform since different inputs from search space may incur different costs for function evaluations. We introduce a cost-aware multi-objective Bayesian optimisation with non-uniform evaluation cost over objective functions by defining cost-aware constraints over the search space. The cost-aware constraints are a sorted tuple of indexes that demonstrate the ordering of dimensions of the search space based on the user's prior knowledge about their cost of usage. We formulate a new multi-objective Bayesian optimisation acquisition function with detailed analysis of the convergence that incorporates this cost-aware constraints while optimising the objective functions. We demonstrate our algorithm based on synthetic and real-world problems in hyperparameter tuning of neural networks and random forests.

References in corpus (5)

• Adam: A Method for Stochastic Optimization
• A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning
• A Literature Survey of Benchmark Functions For Global Optimization Problems
• Predictive Entropy Search for Multi-objective Bayesian Optimization with Constraints
• Bayesian Optimization with Directionally Constrained Search

Cited by in corpus (5)

• A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
• Cost-aware Bayesian Optimization
• Pareto-efficient Acquisition Functions for Cost-Aware Bayesian Optimization
• A Nonmyopic Approach to Cost-Constrained Bayesian Optimization
• Bayesian optimization for modular black-box systems with switching costs