New Heuristics for Parallel and Scalable Bayesian Optimization

Abstract

Bayesian optimization has emerged as a strong candidate tool for global optimization of functions with expensive evaluation costs. However, due to the dynamic nature of research in Bayesian approaches, and the evolution of computing technology, using Bayesian optimization in a parallel computing environment remains a challenge for the non-expert. In this report, I review the state-of-the-art in parallel and scalable Bayesian optimization methods. In addition, I propose practical ways to avoid a few of the pitfalls of Bayesian optimization, such as oversampling of edge parameters and over-exploitation of high performance parameters. Finally, I provide relatively simple, heuristic algorithms, along with their open source software implementations, that can be immediately and easily deployed in any computing environment.

13 pages

References in corpus (7)

• Practical Bayesian Optimization of Machine Learning Algorithms
• Further Optimal Regret Bounds for Thompson Sampling
• Practical Bayesian Optimization for Model Fitting with Bayesian Adaptive Direct Search
• Parallel and Distributed Thompson Sampling for Large-scale Accelerated Exploration of Chemical Space
• An Entropy Search Portfolio for Bayesian Optimization
• Stochastic Bouncy Particle Sampler
• Adaptive Rate of Convergence of Thompson Sampling for Gaussian Process Optimization