Scalable Constrained Bayesian Optimization

Abstract

The global optimization of a high-dimensional black-box function under black-box constraints is a pervasive task in machine learning, control, and engineering. These problems are challenging since the feasible set is typically non-convex and hard to find, in addition to the curses of dimensionality and the heterogeneity of the underlying functions. In particular, these characteristics dramatically impact the performance of Bayesian optimization methods, that otherwise have become the de facto standard for sample-efficient optimization in unconstrained settings, leaving practitioners with evolutionary strategies or heuristics. We propose the scalable constrained Bayesian optimization (SCBO) algorithm that overcomes the above challenges and pushes the applicability of Bayesian optimization far beyond the state-of-the-art. A comprehensive experimental evaluation demonstrates that SCBO achieves excellent results on a variety of benchmarks. To this end, we propose two new control problems that we expect to be of independent value for the scientific community.

arXiv commentTo appear in Proceedings of AISTATS 2021

References in corpus (15)

• Practical Bayesian Optimization of Machine Learning Algorithms
• GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration
• Predictive Entropy Search for Efficient Global Optimization of Black-box Functions
• A Tutorial on Bayesian Optimization
• Bayesian Optimization with Unknown Constraints
• Scalable Global Optimization via Local Bayesian Optimization
• Batched Large-scale Bayesian Optimization in High-dimensional Spaces
• Input Warping for Bayesian Optimization of Non-stationary Functions
• Parallel and Distributed Thompson Sampling for Large-scale Accelerated Exploration of Chemical Space
• BOCK : Bayesian Optimization with Cylindrical Kernels
• Bayesian Optimization of Combinatorial Structures
• Bayesian optimization under mixed constraints with a slack-variable augmented Lagrangian
• Scaling Gaussian Process Regression with Derivatives
• Combinatorial Bayesian Optimization using the Graph Cartesian Product
• High-Dimensional Bayesian Optimization via Additive Models with Overlapping Groups

Cited by in corpus (5)

• c-TPE: Tree-structured Parzen Estimator with Inequality Constraints for Expensive Hyperparameter Optimization
• Failure-averse Active Learning for Physics-constrained Systems
• Parallel Hyperparameter Optimization Of Spiking Neural Network
• Bayesian Optimization Algorithms for Accelerator Physics
• Constrained, Global Optimization of Functions with Lipschitz Continuous Gradients