Reducing The Search Space For Hyperparameter Optimization Using Group Sparsity
arXiv:1904.11095
Abstract
We propose a new algorithm for hyperparameter selection in machine learning algorithms. The algorithm is a novel modification of Harmonica, a spectral hyperparameter selection approach using sparse recovery methods. In particular, we show that a special encoding of hyperparameter space enables a natural group-sparse recovery formulation, which when coupled with HyperBand (a multi-armed bandit strategy) leads to improvement over existing hyperparameter optimization methods such as Successive Halving and Random Search. Experimental results on image datasets such as CIFAR-10 confirm the benefits of our approach.
Published at ICASSP 2019
References in corpus (4)
- Practical Bayesian Optimization of Machine Learning Algorithms
- Efficient Hyperparameter Optimization of Deep Learning Algorithms Using Deterministic RBF Surrogates
- Combination of Hyperband and Bayesian Optimization for Hyperparameter Optimization in Deep Learning
- Hyperparameter Optimization: A Spectral Approach