TransBO: Hyperparameter Optimization via Two-Phase Transfer Learning

Abstract

With the extensive applications of machine learning models, automatic hyperparameter optimization (HPO) has become increasingly important. Motivated by the tuning behaviors of human experts, it is intuitive to leverage auxiliary knowledge from past HPO tasks to accelerate the current HPO task. In this paper, we propose TransBO, a novel two-phase transfer learning framework for HPO, which can deal with the complementary nature among source tasks and dynamics during knowledge aggregation issues simultaneously. This framework extracts and aggregates source and target knowledge jointly and adaptively, where the weights can be learned in a principled manner. The extensive experiments, including static and dynamic transfer learning settings and neural architecture search, demonstrate the superiority of TransBO over the state-of-the-arts.

9 pages and 2 extra pages of appendix

References in corpus (8)

• Practical Bayesian Optimization of Machine Learning Algorithms
• OpenML: networked science in machine learning
• Multi-fidelity Bayesian Optimisation with Continuous Approximations
• OpenBox: A Generalized Black-box Optimization Service
• Efficient Automatic CASH via Rising Bandits
• Efficient End-to-End AutoML via Scalable Search Space Decomposition
• VolcanoML: Speeding up End-to-End AutoML via Scalable Search Space Decomposition
• Hyper-Tune: Towards Efficient Hyper-parameter Tuning at Scale

Cited by in corpus (2)

• Towards General and Efficient Online Tuning for Spark
• Continual Human-in-the-Loop Optimization