Surrogate Model Based Hyperparameter Tuning for Deep Learning with SPOT
arXiv:2105.14625
Abstract
A surrogate model based hyperparameter tuning approach for deep learning is presented. This article demonstrates how the architecture-level parameters (hyperparameters) of deep learning models that were implemented in Keras/tensorflow can be optimized. The implementation of the tuning procedure is 100% accessible from R, the software environment for statistical computing. With a few lines of code, existing R packages (tfruns and SPOT) can be combined to perform hyperparameter tuning. An elementary hyperparameter tuning task (neural network and the MNIST data) is used to exemplify this approach
arXiv commentversion 3
References in corpus (8)
- TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
- Practical Bayesian Optimization of Machine Learning Algorithms
- Neural Architecture Search with Reinforcement Learning
- OpenML: networked science in machine learning
- AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data
- Benchmarking in Optimization: Best Practice and Open Issues
- Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers
- In a Nutshell -- The Sequential Parameter Optimization Toolbox