A resource-efficient method for repeated HPO and NAS problems
arXiv:2103.16111
Abstract
In this work we consider the problem of repeated hyperparameter and neural architecture search (HNAS). We propose an extension of Successive Halving that is able to leverage information gained in previous HNAS problems with the goal of saving computational resources. We empirically demonstrate that our solution is able to drastically decrease costs while maintaining accuracy and being robust to negative transfer. Our method is significantly simpler than competing transfer learning approaches, setting a new baseline for transfer learning in HNAS.
arXiv commentAccepted at AutoML workshop @ ICML 2021
References in corpus (6)
- Scikit-learn: Machine Learning in Python
- Coresets via Bilevel Optimization for Continual Learning and Streaming
- Learning search spaces for Bayesian optimization: Another view of hyperparameter transfer learning
- Tabular Benchmarks for Joint Architecture and Hyperparameter Optimization
- Model-based Asynchronous Hyperparameter and Neural Architecture Search
- Hyperparameter Transfer Across Developer Adjustments