Solving Black-Box Optimization Challenge via Learning Search Space Partition for Local Bayesian Optimization

Abstract

Black-box optimization is one of the vital tasks in machine learning, since it approximates real-world conditions, in that we do not always know all the properties of a given system, up to knowing almost nothing but the results. This paper describes our approach to solving the black-box optimization challenge at NeurIPS 2020 through learning search space partition for local Bayesian optimization. We describe the task of the challenge as well as our algorithm for low budget optimization that we named \texttt{SPBOpt}. We optimize the hyper-parameters of our algorithm for the competition finals using multi-task Bayesian optimization on results from the first two evaluation settings. Our approach has ranked third in the competition finals.

arXiv commentAccepted to Proceedings of Machine Learning Research: NeurIPS 2020 Competition and Demonstration Track

References in corpus (6)

• Practical Bayesian Optimization of Machine Learning Algorithms
• Scalable Bayesian Optimization Using Deep Neural Networks
• Scalable Global Optimization via Local Bayesian Optimization
• pySOT and POAP: An event-driven asynchronous framework for surrogate optimization
• Sample-Efficient Neural Architecture Search by Learning Action Space
• Learning Search Space Partition for Black-box Optimization using Monte Carlo Tree Search

Cited by in corpus (2)

• Bayesian Optimization is Superior to Random Search for Machine Learning Hyperparameter Tuning: Analysis of the Black-Box Optimization Challenge 2020
• Sample-Efficient Neural Architecture Search by Learning Action Space