Improving Hyperparameter Optimization by Planning Ahead

Abstract

Hyperparameter optimization (HPO) is generally treated as a bi-level optimization problem that involves fitting a (probabilistic) surrogate model to a set of observed hyperparameter responses, e.g. validation loss, and consequently maximizing an acquisition function using a surrogate model to identify good hyperparameter candidates for evaluation. The choice of a surrogate and/or acquisition function can be further improved via knowledge transfer across related tasks. In this paper, we propose a novel transfer learning approach, defined within the context of model-based reinforcement learning, where we represent the surrogate as an ensemble of probabilistic models that allows trajectory sampling. We further propose a new variant of model predictive control which employs a simple look-ahead strategy as a policy that optimizes a sequence of actions, representing hyperparameter candidates to expedite HPO. Our experiments on three meta-datasets comparing to state-of-the-art HPO algorithms including a model-free reinforcement learning approach show that the proposed method can outperform all baselines by exploiting a simple planning-based policy.

References in corpus (9)

• The Marginal Value of Adaptive Gradient Methods in Machine Learning
• Scalable Bayesian Optimization Using Deep Neural Networks
• Differentiable MPC for End-to-end Planning and Control
• Model-Based Reinforcement Learning via Meta-Policy Optimization
• Hyp-RL : Hyperparameter Optimization by Reinforcement Learning
• Model-Based Planning with Discrete and Continuous Actions
• Deep Kernel Learning
• A Quantile-based Approach for Hyperparameter Transfer Learning
• Hyperparameter Optimization with Differentiable Metafeatures