112 citations · 228 across the 11 of their papers we have counts for
4 papers · 1 filter
Factored Verification: Detecting and Reducing Hallucination in Summaries of Academic Papers
Charlie George, Andreas Stuhlmüller
Hallucination plagues even frontier LLMs--but how bad is it really for summarizing academic papers? We evaluate Factored Verification, a simple automated method for detecting hallu…
Iterated Decomposition: Improving Science Q&A by Supervising Reasoning Processes
Justin Reppert, Ben Rachbach, Charlie George +4
Language models (LMs) can perform complex reasoning either end-to-end, with hidden latent state, or compositionally, with transparent intermediate state. Composition offers benefit…
RAFT: A Real-World Few-Shot Text Classification Benchmark
Neel Alex, Eli Lifland, Lewis Tunstall +9
Large pre-trained language models have shown promise for few-shot learning, completing text-based tasks given only a few task-specific examples. Will models soon solve classificati…
Evaluating Compositionality in Sentence Embeddings
Ishita Dasgupta, Demi Guo, Andreas Stuhlmüller +2
An important challenge for human-like AI is compositional semantics. Recent research has attempted to address this by using deep neural networks to learn vector space embeddings of…