Better call Surrogates: A hybrid Evolutionary Algorithm for Hyperparameter optimization

Abstract

In this paper, we propose a surrogate-assisted evolutionary algorithm (EA) for hyperparameter optimization of machine learning (ML) models. The proposed STEADE model initially estimates the objective function landscape using RadialBasis Function interpolation, and then transfers the knowledge to an EA technique called Differential Evolution that is used to evolve new solutions guided by a Bayesian optimization framework. We empirically evaluate our model on the hyperparameter optimization problems as a part of the black box optimization challenge at NeurIPS 2020 and demonstrate the improvement brought about by STEADE over the vanilla EA.

Accepted at the black box optimization challenge at NeurIPS 2020

References in corpus (6)

• Practical Bayesian Optimization of Machine Learning Algorithms
• A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning
• BoTorch: A Framework for Efficient Monte-Carlo Bayesian Optimization
• Input Warping for Bayesian Optimization of Non-stationary Functions
• pySOT and POAP: An event-driven asynchronous framework for surrogate optimization
• Maximizing acquisition functions for Bayesian optimization

Cited by in corpus (1)

• Bayesian Optimization is Superior to Random Search for Machine Learning Hyperparameter Tuning: Analysis of the Black-Box Optimization Challenge 2020