Re-Examining Linear Embeddings for High-Dimensional Bayesian Optimization
arXiv:2001.11659
Abstract
Bayesian optimization (BO) is a popular approach to optimize expensive-to-evaluate black-box functions. A significant challenge in BO is to scale to high-dimensional parameter spaces while retaining sample efficiency. A solution considered in existing literature is to embed the high-dimensional space in a lower-dimensional manifold, often via a random linear embedding. In this paper, we identify several crucial issues and misconceptions about the use of linear embeddings for BO. We study the properties of linear embeddings from the literature and show that some of the design choices in current approaches adversely impact their performance. We show empirically that properly addressing these issues significantly improves the efficacy of linear embeddings for BO on a range of problems, including learning a gait policy for robot locomotion.
References in corpus (9)
- Practical Bayesian Optimization of Machine Learning Algorithms
- NAS-Bench-101: Towards Reproducible Neural Architecture Search
- BoTorch: A Framework for Efficient Monte-Carlo Bayesian Optimization
- Scalable Global Optimization via Local Bayesian Optimization
- Batched Large-scale Bayesian Optimization in High-dimensional Spaces
- BOCK : Bayesian Optimization with Cylindrical Kernels
- Scaling Gaussian Process Regression with Derivatives
- Re-Examining Linear Embeddings for High-Dimensional Bayesian Optimization
- High-Dimensional Bayesian Optimization via Additive Models with Overlapping Groups
Cited by in corpus (9)
- Investigating Bayesian optimization for expensive-to-evaluate black box functions: Application in fluid dynamics
- Re-Examining Linear Embeddings for High-Dimensional Bayesian Optimization
- EXPObench: Benchmarking Surrogate-based Optimisation Algorithms on Expensive Black-box Functions
- Bayesian Algorithm Execution: Estimating Computable Properties of Black-box Functions Using Mutual Information
- Computationally Efficient High-Dimensional Bayesian Optimization via Variable Selection
- Comparison of High-Dimensional Bayesian Optimization Algorithms on BBOB
- MBORE: Multi-objective Bayesian Optimisation by Density-Ratio Estimation
- Randomized Maximum Likelihood via High-Dimensional Bayesian Optimization
- High-Dimensional Bayesian Optimization via Semi-Supervised Learning with Optimized Unlabeled Data Sampling