5 papers · 1 filter
Empty Shelves or Lost Keys? Recall Is the Bottleneck for Parametric Factuality
Nitay Calderon, Eyal Ben-David, Zorik Gekhman +2
Standard factuality evaluations of LLMs treat all errors alike, obscuring whether failures arise from missing knowledge (empty shelves) or from limited access to encoded facts (los…
Thinking to Recall: How Reasoning Unlocks Parametric Knowledge in LLMs
Zorik Gekhman, Roee Aharoni, Eran Ofek +3
While reasoning in LLMs plays a natural role in math, code generation, and multi-hop factual questions, its effect on simple, single-hop factual questions remains unclear. Such que…
The FACTS Leaderboard: A Comprehensive Benchmark for Large Language Model Factuality
Aileen Cheng, Alon Jacovi, Amir Globerson +62
We introduce The FACTS Leaderboard, an online leaderboard suite and associated set of benchmarks that comprehensively evaluates the ability of language models to generate factually…
Confidence Improves Self-Consistency in LLMs
Amir Taubenfeld, Tom Sheffer, Eran Ofek +4
Self-consistency decoding enhances LLMs' performance on reasoning tasks by sampling diverse reasoning paths and selecting the most frequent answer. However, it is computationally e…
DRAGged into Conflicts: Detecting and Addressing Conflicting Sources in Search-Augmented LLMs
Arie Cattan, Alon Jacovi, Ori Ram +6
Retrieval Augmented Generation (RAG) is a commonly used approach for enhancing large language models (LLMs) with relevant and up-to-date information. However, the retrieved sources…