On Evaluating LLM Alignment by Evaluating LLMs as Judges

Yixin Liu, Pengfei Liu, Arman Cohan

Alignment with human preferences is an important evaluation aspect of LLMs, requiring them to be helpful, honest, safe, and to precisely follow human instructions. Evaluating large…