Formalized Information Needs Improve Large-Language-Model Relevance Judgments

Jüri Keller, Maik Fröbe, Björn Engelmann +4

Cranfield-style retrieval evaluations with too few or too many relevant documents or with low inter-assessor agreement on relevance can reduce the reliability of observations. In e…