4 papers
Human-Anchored Factuality Evaluation with Strategic Annotation
Yu Wang, Craig Erickson, Kevin Small
LLM-based factuality judges provide scalable evaluation signals, but their metrics are often systematically biased relative to human judgments. We study human-anchored factuality e…
Evaluating the Utility of Grounding Documents with Reference-Free LLM-based Metrics
Yilun Hua, Giuseppe Castellucci, Peter Schulam +2
Retrieval Augmented Generation (RAG)'s success depends on the utility the LLM derives from the content used for grounding. Quantifying content utility does not have a definitive sp…
WINELL: Wikipedia Never-Ending Updating with LLM Agents
Revanth Gangi Reddy, Tanay Dixit, Jiaxin Qin +7
Wikipedia, a vast and continuously consulted knowledge base, faces significant challenges in maintaining up-to-date content due to its reliance on manual human editors. Inspired by…
Towards Better Generalization in Open-Domain Question Answering by Mitigating Context Memorization
Zixuan Zhang, Revanth Gangi Reddy, Kevin Small +2
Open-domain Question Answering (OpenQA) aims at answering factual questions with an external large-scale knowledge corpus. However, real-world knowledge is not static; it updates a…