Learning What Evaluators Value: A Reliable Approach to Modeling Evaluator Preferences

Madeline Celi Kitch, Nihar B. Shah

In many applications, human and LLM evaluators use assessments of relevant criteria to create an overall evaluation for an item or individual. For example, in admissions, committee…