Bayesian Optimization with Exponential Convergence

Abstract

This paper presents a Bayesian optimization method with exponential convergence without the need of auxiliary optimization and without the delta-cover sampling. Most Bayesian optimization methods require auxiliary optimization: an additional non-convex global optimization problem, which can be time-consuming and hard to implement in practice. Also, the existing Bayesian optimization method with exponential convergence requires access to the delta-cover sampling, which was considered to be impractical. Our approach eliminates both requirements and achieves an exponential convergence rate.

arXiv commentIn NIPS 2015 (Advances in Neural Information Processing Systems 2015)

References in corpus (4)

• Practical Bayesian Optimization of Machine Learning Algorithms
• High Dimensional Bayesian Optimisation and Bandits via Additive Models
• Bayesian Multi-Scale Optimistic Optimization
• Exponential Regret Bounds for Gaussian Process Bandits with Deterministic Observations

Cited by in corpus (20)

• Generalization in Deep Learning
• Batched Large-scale Bayesian Optimization in High-dimensional Spaces
• Depth Creates No Bad Local Minima
• Depth with Nonlinearity Creates No Bad Local Minima in ResNets
• Effect of Depth and Width on Local Minima in Deep Learning
• Learning Search Space Partition for Black-box Optimization using Monte Carlo Tree Search
• Generalization in Machine Learning via Analytical Learning Theory
• Bayesian optimization of hyper-parameters in reservoir computing
• Global Continuous Optimization with Error Bound and Fast Convergence
• On Lower Bounds for Standard and Robust Gaussian Process Bandit Optimization
• Tight Regret Bounds for Bayesian Optimization in One Dimension
• Advancing Bayesian Optimization: The Mixed-Global-Local (MGL) Kernel and Length-Scale Cool Down
• A Domain-Shrinking based Bayesian Optimization Algorithm with Order-Optimal Regret Performance
• Optimal Order Simple Regret for Gaussian Process Bandits
• A simple parameter-free and adaptive approach to optimization under a minimal local smoothness assumption
• OneAdapt: Fast Configuration Adaptation for Video Analytics Applications via Backpropagation
• Understanding Dynamics of Nonlinear Representation Learning and Its Application
• Fair Clustering Using Antidote Data
• On Hyper-parameter Tuning for Stochastic Optimization Algorithms
• Infinite dimensional optimistic optimisation with applications on physical systems