5 papers · 1 filter
Language Model Embeddings Can Be Sufficient for Bayesian Optimization
Tung Nguyen, Qiuyi Zhang, Bangding Yang +6
Bayesian Optimization is ubiquitous in experimental design and black-box optimization for improving search efficiency. However, most existing approaches rely on regression models w…
The Vizier Gaussian Process Bandit Algorithm
Xingyou Song, Qiuyi Zhang, Chansoo Lee +8
Google Vizier has performed millions of optimizations and accelerated numerous research and production systems at Google, demonstrating the success of Bayesian optimization as a la…
Position: Leverage Foundational Models for Black-Box Optimization
Xingyou Song, Yingtao Tian, Robert Tjarko Lange +3
Undeniably, Large Language Models (LLMs) have stirred an extraordinary wave of innovation in the machine learning research domain, resulting in substantial impact across diverse fi…
OmniPred: Language Models as Universal Regressors
Xingyou Song, Oscar Li, Chansoo Lee +4
Regression is a powerful tool to accurately predict the outcome metric of a system given a set of parameters, but has traditionally been restricted to methods which are only applic…
Gradientless Descent: High-Dimensional Zeroth-Order Optimization
Daniel Golovin, John Karro, Greg Kochanski +3
Zeroth-order optimization is the process of minimizing an objective , given oracle access to evaluations at adaptively chosen inputs . In this paper, we present two simple…