4 papers
Logit-Contribution Scoring Identifies Non-Literal Retrieval Heads
Aryo Pradipta Gema, Beatrice Alex, Pasquale Minervini
In long-context use, large language models frequently synthesize answers from the meaning of a relevant context span rather than literally copy-pasting them. Identifying which atte…
Stabilizing Efficient Reasoning with Step-Level Advantage Selection
Han Wang, Xiaodong Yu, Jialian Wu +4
Large language models (LLMs) achieve strong reasoning performance by allocating substantial computation at inference time, often generating long and verbose reasoning traces. While…
Inverse Scaling in Test-Time Compute
Aryo Pradipta Gema, Alexander Hägele, Runjin Chen +11
We construct evaluation tasks where extending the reasoning length of Large Reasoning Models (LRMs) deteriorates performance, exhibiting an inverse scaling relationship between tes…
DeCoRe: Decoding by Contrasting Retrieval Heads to Mitigate Hallucinations
Aryo Pradipta Gema, Chen Jin, Ahmed Abdulaal +5
Large Language Models (LLMs) often hallucinate, producing unfaithful or factually incorrect outputs by misrepresenting the provided context or incorrectly recalling internal knowle…