Using Distance Correlation for Efficient Bayesian Optimization
arXiv:2102.08993
Abstract
The need to collect data via expensive measurements of black-box functions is prevalent across science, engineering and medicine. As an example, hyperparameter tuning of a large AI model is critical to its predictive performance but is generally time-consuming and unwieldy. Bayesian optimization (BO) is a collection of methods that aim to address this issue by means of Bayesian statistical inference. In this work, we put forward a BO scheme named BDC, which integrates BO with a statistical measure of association of two random variables called Distance Correlation. BDC balances exploration and exploitation automatically, and requires no manual hyperparameter tuning. We evaluate BDC on a range of benchmark tests and observe that it performs on per with popular BO methods such as the expected improvement and max-value entropy search. We also apply BDC to optimization of sequential integral observations of an unknown terrain and confirm its utility.
14 pages. v2: fixed errors
References in corpus (17)
- Practical Bayesian Optimization of Machine Learning Algorithms
- A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning
- Gaussian Process Optimization in the Bandit Setting: No Regret and Experimental Design
- Deep Reinforcement Learning: An Overview
- Predictive Entropy Search for Efficient Global Optimization of Black-box Functions
- A Tutorial on Bayesian Optimization
- Max-value Entropy Search for Efficient Bayesian Optimization
- Bayesian Optimization with Gradients
- Gaussian Process Optimization with Mutual Information
- Scaling Gaussian Process Regression with Derivatives
- Exploiting gradients and Hessians in Bayesian optimization and Bayesian quadrature
- Multi-resolution Multi-task Gaussian Processes
- Spatially Aggregated Gaussian Processes with Multivariate Areal Outputs
- Efficient Poverty Mapping using Deep Reinforcement Learning
- Evaluating the squared-exponential covariance function in Gaussian processes with integral observations
- Gaussian Process Regression for Binned Data
- Deep kernel learning for integral measurements