Multiple Adaptive Bayesian Linear Regression for Scalable Bayesian Optimization with Warm Start
arXiv:1712.02902
Abstract
Bayesian optimization (BO) is a model-based approach for gradient-free black-box function optimization. Typically, BO is powered by a Gaussian process (GP), whose algorithmic complexity is cubic in the number of evaluations. Hence, GP-based BO cannot leverage large amounts of past or related function evaluations, for example, to warm start the BO procedure. We develop a multiple adaptive Bayesian linear regression model as a scalable alternative whose complexity is linear in the number of observations. The multiple Bayesian linear regression models are coupled through a shared feedforward neural network, which learns a joint representation and transfers knowledge across machine learning problems.
References in corpus (5)
- Adam: A Method for Stochastic Optimization
- MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems
- OpenML: networked science in machine learning
- Parallel and Distributed Thompson Sampling for Large-scale Accelerated Exploration of Chemical Space
- Auto-Differentiating Linear Algebra
Cited by in corpus (7)
- Automated Machine Learning: State-of-The-Art and Open Challenges
- Towards Green Automated Machine Learning: Status Quo and Future Directions
- Continual Learning in Practice
- Auto-Differentiating Linear Algebra
- Amazon SageMaker Autopilot: a white box AutoML solution at scale
- LightAutoML: AutoML Solution for a Large Financial Services Ecosystem
- Uncertainty Quantification and Software Risk Analysis for Digital Twins in the Nearly Autonomous Management and Control Systems: A Review