Hyperparameter Learning via Distributional Transfer
arXiv:1810.06305
Abstract
Bayesian optimisation is a popular technique for hyperparameter learning but typically requires initial exploration even in cases where similar prior tasks have been solved. We propose to transfer information across tasks using learnt representations of training datasets used in those tasks. This results in a joint Gaussian process model on hyperparameters and data representations. Representations make use of the framework of distribution embeddings into reproducing kernel Hilbert spaces. The developed method has a faster convergence compared to existing baselines, in some cases requiring only a few evaluations of the target objective.
References in corpus (13)
- Adam: A Method for Stochastic Optimization
- Scikit-learn: Machine Learning in Python
- Distributed Representations of Words and Phrases and their Compositionality
- Practical Bayesian Optimization of Machine Learning Algorithms
- OpenML: networked science in machine learning
- Predictive Entropy Search for Efficient Global Optimization of Black-box Functions
- Domain Generalization by Marginal Transfer Learning
- Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasets
- BOCK : Bayesian Optimization with Cylindrical Kernels
- Parallel Bayesian Global Optimization of Expensive Functions
- Learning to Warm-Start Bayesian Hyperparameter Optimization
- Optimization, fast and slow: optimally switching between local and Bayesian optimization
- Bayesian Approaches to Distribution Regression