Bayesian Optimization for Iterative Learning
arXiv:1909.09593
Abstract
The performance of deep (reinforcement) learning systems crucially depends on the choice of hyperparameters. Their tuning is notoriously expensive, typically requiring an iterative training process to run for numerous steps to convergence. Traditional tuning algorithms only consider the final performance of hyperparameters acquired after many expensive iterations and ignore intermediate information from earlier training steps. In this paper, we present a Bayesian optimization (BO) approach which exploits the iterative structure of learning algorithms for efficient hyperparameter tuning. We propose to learn an evaluation function compressing learning progress at any stage of the training process into a single numeric score according to both training success and stability. Our BO framework is then balancing the benefit of assessing a hyperparameter setting over additional training steps against their computation cost. We further increase model efficiency by selectively including scores from different training steps for any evaluated hyperparameter set. We demonstrate the efficiency of our algorithm by tuning hyperparameters for the training of deep reinforcement learning agents and convolutional neural networks. Our algorithm outperforms all existing baselines in identifying optimal hyperparameters in minimal time.
arXiv commentCamera ready NeurIPS 2020
References in corpus (16)
- Practical Bayesian Optimization of Machine Learning Algorithms
- Playing Atari with Deep Reinforcement Learning
- A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning
- Dueling Network Architectures for Deep Reinforcement Learning
- A disciplined approach to neural network hyper-parameters: Part 1 -- learning rate, batch size, momentum, and weight decay
- Predictive Entropy Search for Efficient Global Optimization of Black-box Functions
- A Tutorial on Bayesian Optimization
- Deep Reinforcement Learning that Matters
- Max-value Entropy Search for Efficient Bayesian Optimization
- The Parallel Knowledge Gradient Method for Batch Bayesian Optimization
- Freeze-Thaw Bayesian Optimization
- Bayesian Optimization in AlphaGo
- A General Framework for Multi-fidelity Bayesian Optimization with Gaussian Processes
- Theoretical Analysis of Bayesian Optimisation with Unknown Gaussian Process Hyper-Parameters
- Fast Information-theoretic Bayesian Optimisation
- Optimization, fast and slow: optimally switching between local and Bayesian optimization