LLMs Can Patch Up Missing Relevance Judgments in Evaluation

Shivani Upadhyay, Ehsan Kamalloo, Jimmy Lin

Unjudged documents or holes in information retrieval benchmarks are considered non-relevant in evaluation, yielding no gains in measuring effectiveness. However, these missing judg…