Human Evaluation with Automatic Scores: Separating Sampling and Estimation Gains

Yang Liu, Ran Zou, Zhengtao Gui +1

Automatic judges offer inexpensive information about model outputs, but an evaluation team must decide how to turn that information into fewer human annotations. Scores can guide s…