Opening LLM Judges: Recovering Preference Signals Beyond the Final Verdict

Sourabrata Mukherjee, Sunayana Sitaram

LLM judges are widely used to evaluate model outputs, but their verdicts can be unreliable: a judge may favor the worse answer for its position, length, or other surface features.…