39 citations · 92 across the 11 of their papers we have counts for
11 papers
Language Model Embeddings Can Be Sufficient for Bayesian Optimization
Tung Nguyen, Qiuyi Zhang, Bangding Yang +6
Bayesian Optimization is ubiquitous in experimental design and black-box optimization for improving search efficiency. However, most existing approaches rely on regression models w…
The Vizier Gaussian Process Bandit Algorithm
Xingyou Song, Qiuyi Zhang, Chansoo Lee +8
Google Vizier has performed millions of optimizations and accelerated numerous research and production systems at Google, demonstrating the success of Bayesian optimization as a la…
Position: Leverage Foundational Models for Black-Box Optimization
Xingyou Song, Yingtao Tian, Robert Tjarko Lange +3
Undeniably, Large Language Models (LLMs) have stirred an extraordinary wave of innovation in the machine learning research domain, resulting in substantial impact across diverse fi…
OmniPred: Language Models as Universal Regressors
Xingyou Song, Oscar Li, Chansoo Lee +4
Regression is a powerful tool to accurately predict the outcome metric of a system given a set of parameters, but has traditionally been restricted to methods which are only applic…
Task Selection for AutoML System Evaluation
Jonathan Lorraine, Nihesh Anderson, Chansoo Lee +2
Our goal is to assess if AutoML system changes - i.e., to the search space or hyperparameter optimization - will improve the final model's performance on production tasks. However,…
Pre-training helps Bayesian optimization too
Zi Wang, George E. Dahl, Kevin Swersky +6
Bayesian optimization (BO) has become a popular strategy for global optimization of many expensive real-world functions. Contrary to a common belief that BO is suited to optimizing…
Open Source Vizier: Distributed Infrastructure and API for Reliable and Flexible Blackbox Optimization
Xingyou Song, Sagi Perel, Chansoo Lee +2
Vizier is the de-facto blackbox and hyperparameter optimization service across Google, having optimized some of Google's largest products and research efforts. To operate at the sc…
Towards Learning Universal Hyperparameter Optimizers with Transformers
Yutian Chen, Xingyou Song, Chansoo Lee +9
Meta-learning hyperparameter optimization (HPO) algorithms from prior experiments is a promising approach to improve optimization efficiency over objective functions from a similar…
Pre-trained Gaussian Processes for Bayesian Optimization
Zi Wang, George E. Dahl, Kevin Swersky +5
Bayesian optimization (BO) has become a popular strategy for global optimization of expensive real-world functions. Contrary to a common expectation that BO is suited to optimizing…
Gradientless Descent: High-Dimensional Zeroth-Order Optimization
Daniel Golovin, John Karro, Greg Kochanski +3
Zeroth-order optimization is the process of minimizing an objective , given oracle access to evaluations at adaptively chosen inputs . In this paper, we present two simple…
Hardness of Online Sleeping Combinatorial Optimization Problems
Satyen Kale, Chansoo Lee, Dávid Pál
We show that several online combinatorial optimization problems that admit efficient no-regret algorithms become computationally hard in the sleeping setting where a subset of acti…