Predictive Entropy Search for Multi-objective Bayesian Optimization

Abstract

We present PESMO, a Bayesian method for identifying the Pareto set of multi-objective optimization problems, when the functions are expensive to evaluate. The central idea of PESMO is to choose evaluation points so as to maximally reduce the entropy of the posterior distribution over the Pareto set. Critically, the PESMO multi-objective acquisition function can be decomposed as a sum of objective-specific acquisition functions, which enables the algorithm to be used in \emph{decoupled} scenarios in which the objectives can be evaluated separately and perhaps with different costs. This decoupling capability also makes it possible to identify difficult objectives that require more evaluations. PESMO also offers gains in efficiency, as its cost scales linearly with the number of objectives, in comparison to the exponential cost of other methods. We compare PESMO with other related methods for multi-objective Bayesian optimization on synthetic and real-world problems. The results show that PESMO produces better recommendations with a smaller number of evaluations of the objectives, and that a decoupled evaluation can lead to improvements in performance, particularly when the number of objectives is large.

References in corpus (6)

• Adam: A Method for Stochastic Optimization
• Predictive Entropy Search for Efficient Global Optimization of Black-box Functions
• Bayesian Optimization with Unknown Constraints
• A Bayesian approach to constrained single- and multi-objective optimization
• Parallel Predictive Entropy Search for Batch Global Optimization of Expensive Objective Functions
• Faster Computation of Expected Hypervolume Improvement

Cited by in corpus (42)

• BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning
• Autonomous discovery of battery electrolytes with robotic experimentation and machine-learning
• Predictive Entropy Search for Multi-objective Bayesian Optimization with Constraints
• A Flexible Framework for Multi-Objective Bayesian Optimization using Random Scalarizations
• Learning the Pareto Front with Hypernetworks
• Tempo: Robust and Self-Tuning Resource Management in Multi-tenant Parallel Databases
• Constrained Bayesian Optimization with Max-Value Entropy Search
• Output Space Entropy Search Framework for Multi-Objective Bayesian Optimization
• Multi-Objective Bayesian Optimization over High-Dimensional Search Spaces
• Multi-objective Bayesian Optimization using Pareto-frontier Entropy
• Random Hypervolume Scalarizations for Provable Multi-Objective Black Box Optimization
• Learnable Embedding Space for Efficient Neural Architecture Compression
• Max-value Entropy Search for Multi-Objective Bayesian Optimization with Constraints
• Pareto Front-Diverse Batch Multi-Objective Bayesian Optimization
• Scalable Combinatorial Bayesian Optimization with Tractable Statistical models
• Multi-objective Bayesian optimisation with preferences over objectives
• Parallel Predictive Entropy Search for Multi-objective Bayesian Optimization with Constraints
• Uncertainty aware Search Framework for Multi-Objective Bayesian Optimization with Constraints
• CADET: Debugging and Fixing Misconfigurations using Counterfactual Reasoning
• Multi-objective Asynchronous Successive Halving
• GIBBON: General-purpose Information-Based Bayesian OptimisatioN
• Semi-supervised Embedding Learning for High-dimensional Bayesian Optimization
• Information-Theoretic Multi-Objective Bayesian Optimization with Continuous Approximations
• Latency-Aware Neural Architecture Search with Multi-Objective Bayesian Optimization
• Bayesian Optimization with Directionally Constrained Search
• No-regret Algorithms for Multi-task Bayesian Optimization
• Stepwise Model Selection for Sequence Prediction via Deep Kernel Learning
• Weighting Is Worth the Wait: Bayesian Optimization with Importance Sampling
• Myopic Bayesian Design of Experiments via Posterior Sampling and Probabilistic Programming
• Pareto Self-Supervised Training for Few-Shot Learning
• BORE: Bayesian Optimization by Density-Ratio Estimation
• TEA-DNN: the Quest for Time-Energy-Accuracy Co-optimized Deep Neural Networks
• A General Framework for Constrained Bayesian Optimization using Information-based Search
• Multi-objective Bayesian optimization of ferroelectric materials with interfacial control for memory and energy storage applications
• A Similarity Measure of Gaussian Process Predictive Distributions
• A Hybrid 2-stage Neural Optimization for Pareto Front Extraction
• The Kalai-Smorodinski solution for many-objective Bayesian optimization
• Asymptotic Risk of Bezier Simplex Fitting
• Multi-Fidelity Multi-Objective Bayesian Optimization: An Output Space Entropy Search Approach
• Hypervolume-based Multi-objective Bayesian Optimization with Student-t Processes
• A Particle Filter based Multi-Objective Optimization Algorithm: PFOPS
• Approximate Bayesian Computation of Bézier Simplices