38 citations · 148 across the 39 of their papers we have counts for
43 papers
When Agents Slow Down: Understanding LLM Agents' Test-Time Strategies via Elo-per-token Analysis
Kaiyuan Liu, Qiuyang Mang, Bo Peng +6
Large language model (LLM) agents allocate test-time compute adaptively as they revise solutions, use tools, explore alternatives, and decide when to stop. This test-time strategy…
Data Scarcity and Model Sparsity: Mixtures-of-Experts Overfit More to Repeated Data
Atindra Jha, Margaret Li, Jure Leskovec +2
As the supply of human-written text is exhausted, it has become standard practice to repeat language model training data. Prior work has studied data repetition for densely activat…
Knowledge Distillation During Mid-Training Favors Reasoning over Factual Recall
Jacqueline He, Howard Yen, Shuyue Stella Li +9
Logit-based knowledge distillation (KD) is used to train smaller language models (LMs) via supervision from stronger teachers, but whether its benefits are consistent across traini…
SPADE: Self-Play in Adaptive Synthetic Executable Environments
Bo Liu, Simon Yu, Yiding Jiang +15
Continuous self-improvement requires an ever-expanding pool of self-generated, diverse, adaptive goals. For language agents, existing training environment pools (hand-curated, stat…
Prefix Sliding for efficient test-time scaling
Niklas Muennighoff, Zhengyang Wang, Zeyi Chen +15
Test-time scaling uses extra test-time compute to improve performance, such as letting language models reason longer when solving a problem. As models keep the entire reasoning tra…
Spokes: Optimizing for Diverse Pretraining Data Selection
Clarence Lee, Yejin Choi, Luke Zettlemoyer +2
Diversity plays a critical role in data selection, improving performance under fixed data budgets by reducing redundancy and repetition. However, optimizing for diversity is inhere…