Does Reasoning Improve Psychological Depth in Large Language Models? It Depends on Who's Judging

Ruichen Zheng, Yihe Wang, Fabrice Y Harel-Canada +4

LLM-as-a-Judge evaluators are increasingly used to score open-ended generation, yet a judge's correlation with human ratings on its development set may not guarantee valid measurem…