Predictive Entropy Search for Bayesian Optimization with Unknown Constraints

Abstract

Unknown constraints arise in many types of expensive black-box optimization problems. Several methods have been proposed recently for performing Bayesian optimization with constraints, based on the expected improvement (EI) heuristic. However, EI can lead to pathologies when used with constraints. For example, in the case of decoupled constraints---i.e., when one can independently evaluate the objective or the constraints---EI can encounter a pathology that prevents exploration. Additionally, computing EI requires a current best solution, which may not exist if none of the data collected so far satisfy the constraints. By contrast, information-based approaches do not suffer from these failure modes. In this paper, we present a new information-based method called Predictive Entropy Search with Constraints (PESC). We analyze the performance of PESC and show that it compares favorably to EI-based approaches on synthetic and benchmark problems, as well as several real-world examples. We demonstrate that PESC is an effective algorithm that provides a promising direction towards a unified solution for constrained Bayesian optimization.

References in corpus (2)

• Practical Bayesian Optimization of Machine Learning Algorithms
• Predictive Entropy Search for Efficient Global Optimization of Black-box Functions

Cited by in corpus (26)

• A Tutorial on Bayesian Optimization
• Bayesian Optimization is Superior to Random Search for Machine Learning Hyperparameter Tuning: Analysis of the Black-Box Optimization Challenge 2020
• A Bayesian approach to constrained single- and multi-objective optimization
• Predictive Entropy Search for Multi-objective Bayesian Optimization with Constraints
• Bayesian optimization under mixed constraints with a slack-variable augmented Lagrangian
• Self-Optimizing Grinding Machines using Gaussian Process Models and Constrained Bayesian Optimization
• A Stratified Analysis of Bayesian Optimization Methods
• c-TPE: Tree-structured Parzen Estimator with Inequality Constraints for Expensive Hyperparameter Optimization
• Constrained Bayesian Optimization with Max-Value Entropy Search
• Failure-averse Active Learning for Physics-constrained Systems
• Query-limited Black-box Attacks to Classifiers
• Bayesian Optimisation over Multiple Continuous and Categorical Inputs
• Exploring Bayesian Deep Learning for Urgent Instructor Intervention Need in MOOC Forums
• ProBO: Versatile Bayesian Optimization Using Any Probabilistic Programming Language
• Parallel Predictive Entropy Search for Multi-objective Bayesian Optimization with Constraints
• Bayesian Optimization for Selecting Efficient Machine Learning Models
• Bayesian Optimization for Policy Search via Online-Offline Experimentation
• A New Knowledge Gradient-based Method for Constrained Bayesian Optimization
• Sampling Acquisition Functions for Batch Bayesian Optimization
• Bayesian Optimization with Directionally Constrained Search
• On Batch Bayesian Optimization
• Sequential- and Parallel- Constrained Max-value Entropy Search via Information Lower Bound
• A General Framework for Constrained Bayesian Optimization using Information-based Search
• Domain Knowledge Injection in Bayesian Search for New Materials
• Speed-Constrained Tuning for Statistical Machine Translation Using Bayesian Optimization
• Bayesian Optimization and Deep Learning forsteering wheel angle prediction