Can External Validation Tools Improve Annotation Quality for LLM-as-a-Judge?

Arduin Findeis, Floris Weers, Guoli Yin +3

Pairwise preferences over model responses are widely collected to evaluate and provide feedback to large language models (LLMs). Given two alternative model responses to the same i…