Adaptive and Safe Bayesian Optimization in High Dimensions via One-Dimensional Subspaces

Abstract

Bayesian optimization is known to be difficult to scale to high dimensions, because the acquisition step requires solving a non-convex optimization problem in the same search space. In order to scale the method and keep its benefits, we propose an algorithm (LineBO) that restricts the problem to a sequence of iteratively chosen one-dimensional sub-problems that can be solved efficiently. We show that our algorithm converges globally and obtains a fast local rate when the function is strongly convex. Further, if the objective has an invariant subspace, our method automatically adapts to the effective dimension without changing the algorithm. When combined with the SafeOpt algorithm to solve the sub-problems, we obtain the first safe Bayesian optimization algorithm with theoretical guarantees applicable in high-dimensional settings. We evaluate our method on multiple synthetic benchmarks, where we obtain competitive performance. Further, we deploy our algorithm to optimize the beam intensity of the Swiss Free Electron Laser with up to 40 parameters while satisfying safe operation constraints.

References in corpus (4)

• Max-value Entropy Search for Efficient Bayesian Optimization
• Gaussian Processes and Kernel Methods: A Review on Connections and Equivalences
• A Stochastic Line Search Method with Convergence Rate Analysis
• Lower Bounds on Regret for Noisy Gaussian Process Bandit Optimization

Cited by in corpus (17)

• Towards Dynamic and Safe Configuration Tuning for Cloud Databases
• Multi-Objective Bayesian Optimization for Accelerator Tuning
• Distributionally Robust Bayesian Optimization
• A Bayesian Optimization approach for calibrating large-scale activity-based transport models
• Multi-Objective Bayesian Optimization over High-Dimensional Search Spaces
• The Differentiable Cross-Entropy Method
• Multipoint-BAX: A New Approach for Efficiently Tuning Particle Accelerator Emittance via Virtual Objectives
• Learning to Guide Random Search
• Bayesian Optimization Algorithms for Accelerator Physics
• Projective Preferential Bayesian Optimization
• The Laser-hybrid Accelerator for Radiobiological Applications
• LinEasyBO: Scalable Bayesian Optimization Approach for Analog Circuit Synthesis via One-Dimensional Subspaces
• Sequential Subspace Search for Functional Bayesian Optimization Incorporating Experimenter Intuition
• Modeling of Advanced Accelerator Concepts
• Scalable Bayesian Optimization with Sparse Gaussian Process Models
• Bayesian optimization for modular black-box systems with switching costs
• Introduction to Machine Learning for Accelerator Physics