4 papers
Embedding by Elicitation: Dynamic Representations for Bayesian Optimization of System Prompts
Zhiyuan Jerry Lin, Benjamin Letham, Samuel Dooley +2
System prompts are a central control mechanism in modern AI systems, shaping behavior across conversations, tasks, and user populations. Yet they are difficult to tune when feedbac…
LILO: Bayesian Optimization with Natural Language Feedback
Katarzyna Kobalczyk, Zhiyuan Jerry Lin, Benjamin Letham +3
Many real-world optimization problems are guided by complex, subjective preferences that are difficult to express as explicit closed-form objectives. In response, we introduce Lang…
Experimenting, Fast and Slow: Bayesian Optimization of Long-term Outcomes with Online Experiments
Qing Feng, Samuel Daulton, Benjamin Letham +2
Online experiments in internet systems, also known as A/B tests, are used for a wide range of system tuning problems, such as optimizing recommender system ranking policies and lea…
Robust Gaussian Processes via Relevance Pursuit
Sebastian Ament, Elizabeth Santorella, David Eriksson +3
Gaussian processes (GPs) are non-parametric probabilistic regression models that are popular due to their flexibility, data efficiency, and well-calibrated uncertainty estimates. H…