Batched Large-scale Bayesian Optimization in High-dimensional Spaces

Abstract

Bayesian optimization (BO) has become an effective approach for black-box function optimization problems when function evaluations are expensive and the optimum can be achieved within a relatively small number of queries. However, many cases, such as the ones with high-dimensional inputs, may require a much larger number of observations for optimization. Despite an abundance of observations thanks to parallel experiments, current BO techniques have been limited to merely a few thousand observations. In this paper, we propose ensemble Bayesian optimization (EBO) to address three current challenges in BO simultaneously: (1) large-scale observations; (2) high dimensional input spaces; and (3) selections of batch queries that balance quality and diversity. The key idea of EBO is to operate on an ensemble of additive Gaussian process models, each of which possesses a randomized strategy to divide and conquer. We show unprecedented, previously impossible results of scaling up BO to tens of thousands of observations within minutes of computation.

Proceedings of the 21st International Conference on Artificial Intelligence and Statistics (AISTATS) 2018, Lanzarote, Spain

References in corpus (2)

• Predictive Entropy Search for Efficient Global Optimization of Black-box Functions
• Batched High-dimensional Bayesian Optimization via Structural Kernel Learning

Cited by in corpus (38)

• Accelerating high-throughput virtual screening through molecular pool-based active learning
• Towards Dynamic and Safe Configuration Tuning for Cloud Databases
• ENTMOOT: A Framework for Optimization over Ensemble Tree Models
• Tuning Hyperparameters without Grad Students: Scalable and Robust Bayesian Optimisation with Dragonfly
• Re-Examining Linear Embeddings for High-Dimensional Bayesian Optimization
• Learning Search Space Partition for Black-box Optimization using Monte Carlo Tree Search
• Scalable Constrained Bayesian Optimization
• A Bayesian Optimization approach for calibrating large-scale activity-based transport models
• Think Global and Act Local: Bayesian Optimisation over High-Dimensional Categorical and Mixed Search Spaces
• On Local Optimizers of Acquisition Functions in Bayesian Optimization
• Deep Kernel and Image Quality Estimators for Optimizing Robotic Ultrasound Controller using Bayesian Optimization
• Efficiently Sampling Functions from Gaussian Process Posteriors
• Knowing The What But Not The Where in Bayesian Optimization
• Hyperboost: Hyperparameter Optimization by Gradient Boosting surrogate models
• Optimal Transport Kernels for Sequential and Parallel Neural Architecture Search
• Optimizing with Low Budgets: a Comparison on the Black-box Optimization Benchmarking Suite and OpenAI Gym
• Efficient Model-Based Reinforcement Learning through Optimistic Policy Search and Planning
• Gaussian Process Optimization with Adaptive Sketching: Scalable and No Regret
• Facilitating Database Tuning with Hyper-Parameter Optimization: A Comprehensive Experimental Evaluation
• Bayesian Optimization over Hybrid Spaces
• Pathwise Conditioning of Gaussian Processes
• Scalable Thompson Sampling using Sparse Gaussian Process Models
• Practical Batch Bayesian Optimization for Less Expensive Functions
• Practical Transfer Learning for Bayesian Optimization
• A portfolio approach to massively parallel Bayesian optimization
• Parameter Optimization using high-dimensional Bayesian Optimization
• Scalable Acceleration for Classification-Based Derivative-Free Optimization
• Comparison of High-Dimensional Bayesian Optimization Algorithms on BBOB
• Matérn Gaussian Processes on Graphs
• Efficient Batch Black-box Optimization with Deterministic Regret Bounds
• A Hierarchical Approach to Scaling Batch Active Search Over Structured Data
• RTDK-BO: High Dimensional Bayesian Optimization with Reinforced Transformer Deep kernels
• AutoCP: Automated Pipelines for Accurate Prediction Intervals
• Robot Design With Neural Networks, MILP Solvers and Active Learning
• Robust Policy Search for Robot Navigation
• DEEP-BO for Hyperparameter Optimization of Deep Networks
• Bayesian Optimistic Optimisation with Exponentially Decaying Regret
• Searching in the Forest for Local Bayesian Optimization