4 papers
Recursive Language Models Generalize Out of Domain
Chenxiao Yang, Zhiyuan Li, David McAllester +1
We study when limiting what a language model can see improves learning. We compare standard CoT, the more general learner that reads the full trace, with recursive language models,…
OKBench: Democratizing LLM Evaluation with Fully Automated, On-Demand, Open Knowledge Benchmarking
Yanhong Li, Tianyang Xu, Kenan Tang +3
Knowledge-intensive question answering is central to large language models (LLMs) and is typically assessed using static benchmarks derived from sources like Wikipedia and textbook…
Context-Efficient Retrieval with Factual Decomposition
Yanhong Li, David Yunis, David McAllester +1
There has recently been considerable interest in incorporating information retrieval into large language models (LLMs). Retrieval from a dynamically expanding external corpus of te…
PENCIL: Long Thoughts with Short Memory
Chenxiao Yang, Nathan Srebro, David McAllester +1
While state-of-the-art LLMs have demonstrated great promise of using long Chains-of-Thought (CoT) to boost reasoning, scaling it up to more challenging problems at test-time is fun…