Calibrate, Don't Curate: Label-Efficient Estimation from Noisy LLM Judges

Yanran Li

Multi-judge evaluation is increasingly used to assess LLMs and reward models, and the prevailing heuristic is to curate: keep the most accurate judges and discard weaker ones. We s…