11 papers
GIFT: Gradient Informed Filtering for Distillation Augmented Reasoning RL
Shijie Li, Yilin Gao, Tieyuan Chen +3
Training large language models for complex reasoning tasks relies on two complementary signals: sparse, outcome-grounded reinforcement learning (RL) for correctness, and dense, tok…
LatentLoop: Scaling Looped MoE Models with Asymmetric Compute Allocation
Zefeng He, Yunkai Chai, Xiaoyang Chen +5
Looped Mixture-of-Experts (MoE) models combine parameter reuse with sparse computation to improve performance under matched budgets. Prior work demonstrates that different loop vis…
TPPO: Tracking-Value PPO For Language Model Reasoning
Futing Wang, Chenxi Li, Jianhao Yan +6
Proximal Policy Optimization (PPO) is widely used in reinforcement learning with verifiable rewards for language-model reasoning. While \emph{value evolution along trajectory} char…
Gains and Collapse in On-Policy Distillation:A Reinforcement Learning Perspective
Han Cui, Jianhao Yan, Yun Luo +3
On-policy distillation (OPD) has become an important approach to language model post-training. However, despite its performance gains, OPD can also collapse into excessively long a…
Draft-OPD: On-Policy Distillation for Speculative Draft Models
Haodi Lei, Yafu Li, Haoran Zhang +8
Speculative decoding accelerates large language model inference by pairing a target model with a lightweight draft model whose proposed tokens are verified in parallel. A common wa…
Teaching Thinking Models to Reason with Tools: A Full-Pipeline Recipe for Tool-Integrated Reasoning
Qianjia Cheng, Yuchen Zhang, Zhilin Wang +9
Tool-integrated reasoning (TIR) offers a direct way to extend thinking models beyond the limits of text-only reasoning. Paradoxically, we observe that tool-enabled evaluation can d…
TEMPO: Scaling Test-time Training for Large Reasoning Models
Qingyang Zhang, Xinke Kong, Haitao Wu +7
Test-time training (TTT) adapts model parameters on unlabeled test instances during inference time, which continuously extends capabilities beyond the reach of offline training. De…
Think Longer to Explore Deeper: Learn to Explore In-Context via Length-Incentivized Reinforcement Learning
Futing Wang, Jianhao Yan, Yun Luo +6
Achieving effective test-time scaling requires models to engage in In-Context Exploration -- the intrinsic ability to generate, verify, and refine multiple reasoning hypotheses wit…
Spotlight on Token Perception for Multimodal Reinforcement Learning
Siyuan Huang, Xiaoye Qu, Yafu Li +4
While Reinforcement Learning with Verifiable Rewards (RLVR) has advanced the reasoning capabilities of Large Vision-Language Models (LVLMs), most existing methods in multimodal rea…
Towards an AI Musician: Synthesizing Sheet Music Problems for Musical Reasoning
Zhilin Wang, Zhe Yang, Yun Luo +8
Enhancing the ability of Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs) to interpret sheet music is a crucial step toward building AI musicians. However,…
HiPhO: How Far Are (M)LLMs from Humans in the Latest High School Physics Olympiad Benchmark?
Fangchen Yu, Haiyuan Wan, Qianjia Cheng +14
Recently, the physical capabilities of (M)LLMs have garnered increasing attention. However, existing benchmarks for physics suffer from two major gaps: they neither provide systema…