13 papers
Compositional RL: Evaluating and Training LLMs for Collaborative Reasoning
Shangbin Feng, Yulia Tsvetkov, Jan Kautz +1
Language models are often evaluated alone but increasingly deployed together, routed between, made to debate, ensembled, or collaboratively trained. For a model inside a system, wh…
MagicBeanStalk: Escaping the RLVR Zero Advantage Trap with Recursive Data Synthesis
Ximing Lu, Aviral Kumar, Jan Kautz +2
Data-centric scaling of RLVR has focused on collecting or generating verifiable training problems. However, corpus size alone does not determine whether the dataset provides a usef…
Introspective X Training: Feedback Conditioning Improves Scaling Across all LLM Training Stages
Brandon Cui, Ximing Lu, Jaehun Jung +7
We tackle the question of how to scale more efficiently across the many, ever-growing stages of current LLM training pipelines. Our guiding intuition stems from the fact that the d…
How to Instruct Your Robot: Dense Language Annotations Power Robot Policy Learning
Bosung Kim, Ruiyi Wang, David Acuna +5
Scaling robot policy learning is bottlenecked by the cost of collecting demonstrations, while language annotations for existing demonstrations are comparatively cheap. We study lan…
DeltaPrompts: Escaping the Zero-Delta Trap in Multimodal Distillation
Jaehun Jung, Hyunwoo Kim, Brandon Cui +4
Distillation enables compact Vision-Language Models (VLMs) to obtain strong reasoning capabilities, yet the prompts driving this process are typically chosen via simple heuristics…
Privasis: Synthesizing the Largest "Public" Private Dataset from Scratch
Hyunwoo Kim, Niloofar Mireshghallah, Michael Duan +11
Research involving privacy-sensitive data has always been constrained by data scarcity, standing in sharp contrast to other areas that have benefited from data scaling. This challe…
Golden Goose: A Simple Trick to Synthesize Unlimited RLVR Tasks from Unverifiable Internet Text
Ximing Lu, David Acuna, Jaehun Jung +12
Reinforcement Learning with Verifiable Rewards (RLVR) has become a cornerstone for unlocking complex reasoning in Large Language Models (LLMs). Yet, scaling up RL is bottlenecked b…
Long Grounded Thoughts: Synthesizing Visual Problems and Reasoning Chains at Scale
David Acuna, Chao-Han Huck Yang, Yuntian Deng +6
Despite rapid progress, multimodal reasoning still lacks a systematic approach to synthesize large-scale vision-centric datasets beyond visual math. We introduce a framework able t…
Goedel-Prover-V2: Scaling Formal Theorem Proving with Scaffolded Data Synthesis and Self-Correction
Yong Lin, Shange Tang, Bohan Lyu +17
We introduce Goedel-Prover-V2, a series of open-source language models that set a new state-of-the-art in automated theorem proving. Built on the standard expert iteration and rein…
Socratic-MCTS: Test-Time Visual Reasoning by Asking the Right Questions
David Acuna, Ximing Lu, Jaehun Jung +4
Recent research in vision-language models (VLMs) has centered around the possibility of equipping them with implicit long-form chain-of-thought reasoning -- akin to the success obs…
Prismatic Synthesis: Gradient-based Data Diversification Boosts Generalization in LLM Reasoning
Jaehun Jung, Seungju Han, Ximing Lu +7
Effective generalization in language models depends critically on the diversity of their training data. Yet existing diversity metrics often fall short of this goal, relying on sur…
LongPerceptualThoughts: Distilling System-2 Reasoning for System-1 Perception
Yuan-Hong Liao, Sven Elflein, Liu He +4
Recent reasoning models through test-time scaling have demonstrated that long chain-of-thoughts can unlock substantial performance boosts in hard reasoning tasks such as math and c…
Retro-Search: Exploring Untaken Paths for Deeper and Efficient Reasoning
Ximing Lu, Seungju Han, David Acuna +8
Large reasoning models exhibit remarkable reasoning capabilities via long, elaborate reasoning trajectories. Supervised fine-tuning on such reasoning traces, also known as distilla…