BOCK : Bayesian Optimization with Cylindrical Kernels

Abstract

A major challenge in Bayesian Optimization is the boundary issue (Swersky, 2017) where an algorithm spends too many evaluations near the boundary of its search space. In this paper, we propose BOCK, Bayesian Optimization with Cylindrical Kernels, whose basic idea is to transform the ball geometry of the search space using a cylindrical transformation. Because of the transformed geometry, the Gaussian Process-based surrogate model spends less budget searching near the boundary, while concentrating its efforts relatively more near the center of the search region, where we expect the solution to be located. We evaluate BOCK extensively, showing that it is not only more accurate and efficient, but it also scales successfully to problems with a dimensionality as high as 500. We show that the better accuracy and scalability of BOCK even allows optimizing modestly sized neural network layers, as well as neural network hyperparameters.

arXiv comment10 pages, 5 figures, 5 tables, 1 algorithm

Cited by in corpus (24)

• ENTMOOT: A Framework for Optimization over Ensemble Tree Models
• High-Dimensional Bayesian Optimization with Sparse Axis-Aligned Subspaces
• Re-Examining Linear Embeddings for High-Dimensional Bayesian Optimization
• Revisiting Bayesian Optimization in the light of the COCO benchmark
• Learning Search Space Partition for Black-box Optimization using Monte Carlo Tree Search
• Scalable Constrained Bayesian Optimization
• Hyperparameter Learning via Distributional Transfer
• Multi-Objective Bayesian Optimization over High-Dimensional Search Spaces
• A Framework of Transfer Learning in Object Detection for Embedded Systems
• Bayesian Optimization Meets Riemannian Manifolds in Robot Learning
• Knowing The What But Not The Where in Bayesian Optimization
• High-Dimensional Bayesian Optimisation with Variational Autoencoders and Deep Metric Learning
• Principled analytic classifier for positive-unlabeled learning via weighted integral probability metric
• The Differentiable Cross-Entropy Method
• Efficient Bayesian Experimental Design for Implicit Models
• Incorporating Expert Prior Knowledge into Experimental Design via Posterior Sampling
• A portfolio approach to massively parallel Bayesian optimization
• Attentive Neural Processes and Batch Bayesian Optimization for Scalable Calibration of Physics-Informed Digital Twins
• Go with the Flow: Perception-refined Physics Simulation
• Mixed Variable Bayesian Optimization with Frequency Modulated Kernels
• ExperienceThinking: Constrained Hyperparameter Optimization based on Knowledge and Pruning
• High-Dimensional Bayesian Optimization via Nested Riemannian Manifolds
• A Simple Heuristic for Bayesian Optimization with A Low Budget
• Scaling Bayesian Optimization With Game Theory