Sequential- and Parallel- Constrained Max-value Entropy Search via Information Lower Bound
arXiv:2102.09788
Abstract
Max-value entropy search (MES) is one of the state-of-the-art approaches in Bayesian optimization (BO). In this paper, we propose a novel variant of MES for constrained problems, called Constrained MES via Information lower BOund (CMES-IBO), that is based on a Monte Carlo (MC) estimator of a lower bound of a mutual information (MI). Unlike existing studies, our MI is defined so that uncertainty with respect to feasibility can be incorporated. We derive a lower bound of the MI that guarantees non-negativity, while a constrained counterpart of conventional MES can be negative. We further provide theoretical analysis that assures the low-variability of our estimator which has never been investigated for any existing information-theoretic BO. Moreover, using the conditional MI, we extend CMES-IBO to the parallel setting while maintaining the desirable properties. We demonstrate the effectiveness of CMES-IBO by several benchmark functions and real-world problems.
arXiv comment39pages, 8 figures
References in corpus (10)
- Predictive Entropy Search for Efficient Global Optimization of Black-box Functions
- Bayesian Optimization with Unknown Constraints
- Max-value Entropy Search for Efficient Bayesian Optimization
- Predictive Entropy Search for Bayesian Optimization with Unknown Constraints
- Maximizing acquisition functions for Bayesian optimization
- Moments Calculation For the Doubly Truncated Multivariate Normal Density
- On entropy for mixtures of discrete and continuous variables
- Constrained Bayesian Optimization with Max-Value Entropy Search
- Max-value Entropy Search for Multi-Objective Bayesian Optimization with Constraints
- GIBBON: General-purpose Information-Based Bayesian OptimisatioN