4 papers
ARIA: A Causal-Aware Framework for Rescuing LLM Reasoning in Trustworthy Materials Discovery
Yi Cao, Liaoyaqi Wang, Jieneng Chen +3
Generative models have revolutionized the process of materials discovery, yet they often fail to satisfy underlying physical causality. Through an analysis of Large Language Models…
Beyond Relevance: On the Relationship Between Retrieval and RAG Information Coverage
Saron Samuel, Alexander Martin, Eugene Yang +5
Retrieval-augmented generation (RAG) systems combine document retrieval with a generative model to address complex information seeking tasks like report generation. While the relat…
CoverageBench: Evaluating Information Coverage across Tasks and Domains
Saron Samuel, Andrew Yates, Dawn Lawrie +4
We wish to measure the information coverage of an ad hoc retrieval algorithm, that is, how much of the range of available relevant information is covered by the search results. Inf…
LLMs Provide Unstable Answers to Legal Questions
Andrew Blair-Stanek, Benjamin Van Durme
An LLM is stable if it reaches the same conclusion when asked the identical question multiple times. We find leading LLMs like gpt-4o, claude-3.5, and gemini-1.5 are unstable when…