6 papers
CollabEval: Statistically Efficient Collaborative Model Evaluation via Matrix Completion
Adam Fisch, Daniel Deutsch, Joshua Maynez +5
Evaluating generative AI models is a routine, but resource-intensive, process that is conducted over and over again during the course of model development. In this work, we propose…
Multiple-Prediction-Powered Inference
Charlie Cowen-Breen, Alekh Agarwal, Stephen Bates +4
Statistical estimation often involves tradeoffs between expensive, high-quality measurements and a variety of lower-quality proxies. We introduce Multiple-Prediction-Powered Infere…
Cost-Optimal Active AI Model Evaluation
Anastasios N. Angelopoulos, Jacob Eisenstein, Jonathan Berant +2
The development lifecycle of generative AI systems requires continual evaluation, data acquisition, and annotation, which is costly in both resources and time. In practice, rapid i…
Theoretical guarantees on the best-of-n alignment policy
Ahmad Beirami, Alekh Agarwal, Jonathan Berant +4
A simple and effective method for the inference-time alignment and scaling test-time compute of generative models is best-of- sampling, where samples are drawn from a refere…
Robust Preference Optimization through Reward Model Distillation
Adam Fisch, Jacob Eisenstein, Vicky Zayats +5
Language model (LM) post-training (or alignment) involves maximizing a reward function that is derived from preference annotations. Direct Preference Optimization (DPO) is a popula…
Optimizing Pre-Training Data Mixtures with Mixtures of Data Expert Models
Lior Belenki, Alekh Agarwal, Tianze Shi +1
We propose a method to optimize language model pre-training data mixtures through efficient approximation of the cross-entropy loss corresponding to each candidate mixture via a Mi…