Experimental Investigation and Evaluation of Model-based Hyperparameter Optimization
arXiv:2107.08761
Abstract
Machine learning algorithms such as random forests or xgboost are gaining more importance and are increasingly incorporated into production processes in order to enable comprehensive digitization and, if possible, automation of processes. Hyperparameters of these algorithms used have to be set appropriately, which can be referred to as hyperparameter tuning or optimization. Based on the concept of tunability, this article presents an overview of theoretical and practical results for popular machine learning algorithms. This overview is accompanied by an experimental analysis of 30 hyperparameters from six relevant machine learning algorithms. In particular, it provides (i) a survey of important hyperparameters, (ii) two parameter tuning studies, and (iii) one extensive global parameter tuning study, as well as (iv) a new way, based on consensus ranking, to analyze results from multiple algorithms. The R package mlr is used as a uniform interface to the machine learning models. The R package SPOT is used to perform the actual tuning (optimization). All additional code is provided together with this paper.
References in corpus (6)
- OpenML: networked science in machine learning
- Performance evaluation and hyperparameter tuning of statistical and machine-learning models using spatial data
- Greed is Good: Exploration and Exploitation Trade-offs in Bayesian Optimisation
- To tune or not to tune the number of trees in random forest?
- An Experimental Comparison of Old and New Decision Tree Algorithms
- In a Nutshell -- The Sequential Parameter Optimization Toolbox