Graders should cheat: privileged information enables expert-level automated evaluations

Jin Peng Zhou, Sébastien M. R. Arnold, Nan Ding +3

Auto-evaluating language models (LMs), i.e., using a grader LM to evaluate the candidate LM, is an appealing way to accelerate the evaluation process and the cost associated with i…