3 papers
cs.LG2026
CollabEval: Statistically Efficient Collaborative Model Evaluation via Matrix Completion
Adam Fisch, Daniel Deutsch, Joshua Maynez +5
Evaluating generative AI models is a routine, but resource-intensive, process that is conducted over and over again during the course of model development. In this work, we propose…
math.ST2026
Multiple-Prediction-Powered Inference
Charlie Cowen-Breen, Alekh Agarwal, Stephen Bates +4
Statistical estimation often involves tradeoffs between expensive, high-quality measurements and a variety of lower-quality proxies. We introduce Multiple-Prediction-Powered Infere…
cs.LG2024
Stratified Prediction-Powered Inference for Hybrid Language Model Evaluation
Adam Fisch, Joshua Maynez, R. Alex Hofer +3
Prediction-powered inference (PPI) is a method that improves statistical estimates based on limited human-labeled data. PPI achieves this by combining small amounts of human-labele…