Heteroscedastic Treed Bayesian Optimisation

Abstract

Optimising black-box functions is important in many disciplines, such as tuning machine learning models, robotics, finance and mining exploration. Bayesian optimisation is a state-of-the-art technique for the global optimisation of black-box functions which are expensive to evaluate. At the core of this approach is a Gaussian process prior that captures our belief about the distribution over functions. However, in many cases a single Gaussian process is not flexible enough to capture non-stationarity in the objective function. Consequently, heteroscedasticity negatively affects performance of traditional Bayesian methods. In this paper, we propose a novel prior model with hierarchical parameter learning that tackles the problem of non-stationarity in Bayesian optimisation. Our results demonstrate substantial improvements in a wide range of applications, including automatic machine learning and mining exploration.

References in corpus (6)

• Practical Bayesian Optimization of Machine Learning Algorithms
• A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning
• Narrowing the Gap: Random Forests In Theory and In Practice
• Hybrid Batch Bayesian Optimization
• An Entropy Search Portfolio for Bayesian Optimization
• Theoretical Analysis of Bayesian Optimisation with Unknown Gaussian Process Hyper-Parameters

Cited by in corpus (10)

• Information Directed Sampling and Bandits with Heteroscedastic Noise
• Bayesian Optimization with Adaptive Kernels for Robot Control
• Noisy-Input Entropy Search for Efficient Robust Bayesian Optimization
• Achieving Robustness to Aleatoric Uncertainty with Heteroscedastic Bayesian Optimisation
• Advancing Bayesian Optimization: The Mixed-Global-Local (MGL) Kernel and Length-Scale Cool Down
• Stochastic Learning of Nonstationary Kernels for Natural Language Modeling
• Neural fidelity warping for efficient robot morphology design
• Kernelized Hashcode Representations for Relation Extraction
• Funneled Bayesian Optimization for Design, Tuning and Control of Autonomous Systems
• Robust Policy Search for Robot Navigation