6 papers
POPE: Learning to Reason on Hard Problems via Privileged On-Policy Exploration
Yuxiao Qu, Amrith Setlur, Virginia Smith +2
Reinforcement learning (RL) has improved the reasoning abilities of large language models (LLMs), yet state-of-the-art methods still fail to learn on many training problems. On har…
CaRT: Teaching LLM Agents to Know When They Know Enough
Grace Liu, Yuxiao Qu, Jeff Schneider +2
Many tasks require learned models to strategically gather relevant information over multiple rounds of interaction before actually acting on a task. Strategic information gathering…
RLAD: Training LLMs to Discover Abstractions for Solving Reasoning Problems
Yuxiao Qu, Anikait Singh, Yoonho Lee +4
Reasoning requires going beyond pattern matching or memorization of solutions to identify and implement "algorithmic procedures" that can be used to deduce answers to hard problems…
e3: Learning to Explore Enables Extrapolation of Test-Time Compute for LLMs
Amrith Setlur, Matthew Y. R. Yang, Charlie Snell +5
Test-time scaling offers a promising path to improve LLM reasoning by utilizing more compute at inference time; however, the true promise of this paradigm lies in extrapolation (i.…
Thinking vs. Doing: Agents that Reason by Scaling Test-Time Interaction
Junhong Shen, Hao Bai, Lunjun Zhang +8
The current paradigm of test-time scaling relies on generating long reasoning traces ("thinking" more) before producing a response. In agent problems that require interaction, this…
Optimizing Test-Time Compute via Meta Reinforcement Fine-Tuning
Yuxiao Qu, Matthew Y. R. Yang, Amrith Setlur +4
Training models to effectively use test-time compute is crucial for improving the reasoning performance of LLMs. Current methods mostly do so via fine-tuning on search traces or ru…