Meta-Learning Acquisition Functions for Transfer Learning in Bayesian Optimization

Abstract

Transferring knowledge across tasks to improve data-efficiency is one of the open key challenges in the field of global black-box optimization. Readily available algorithms are typically designed to be universal optimizers and, therefore, often suboptimal for specific tasks. We propose a novel transfer learning method to obtain customized optimizers within the well-established framework of Bayesian optimization, allowing our algorithm to utilize the proven generalization capabilities of Gaussian processes. Using reinforcement learning to meta-train an acquisition function (AF) on a set of related tasks, the proposed method learns to extract implicit structural information and to exploit it for improved data-efficiency. We present experiments on a simulation-to-real transfer task as well as on several synthetic functions and on two hyperparameter search problems. The results show that our algorithm (1) automatically identifies structural properties of objective functions from available source tasks or simulations, (2) performs favourably in settings with both scarse and abundant source data, and (3) falls back to the performance level of general AFs if no particular structure is present.

References in corpus (13)

• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Practical Bayesian Optimization of Machine Learning Algorithms
• A Brief Survey of Deep Reinforcement Learning
• A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning
• High-Dimensional Continuous Control Using Generalized Advantage Estimation
• Theoretical Models of Learning to Learn
• On First-Order Meta-Learning Algorithms
• Predictive Entropy Search for Efficient Global Optimization of Black-box Functions
• Learning to Learn without Gradient Descent by Gradient Descent
• Learning to Optimize
• Learning to Optimize Neural Nets
• Transferring Knowledge across Learning Processes
• Goal-Driven Dynamics Learning via Bayesian Optimization

Cited by in corpus (6)

• Apollo: Transferable Architecture Exploration
• The Differentiable Cross-Entropy Method
• Meta-Surrogate Benchmarking for Hyperparameter Optimization
• HPO-B: A Large-Scale Reproducible Benchmark for Black-Box HPO based on OpenML
• Optimizing Discrete Spaces via Expensive Evaluations: A Learning to Search Framework
• JUMBO: Scalable Multi-task Bayesian Optimization using Offline Data