8 papers
Dynamic Rollout Editing for Reducing Overthinking in RL-Trained Reasoning Models
Zihao Wei, Wenjie Shi, Liang Pang +8
Long-form chain-of-thought reasoning can improve LLM performance on complex tasks, but models often continue generating unnecessary reasoning after a correct answer has emerged. We…
Teacher-Guided Policy Optimization for On-Policy Reasoning Distillation under Large Policy Divergence
Xinyu Liu, Kechen Jiao, Chunyang Xiao +10
On-policy distillation (OPD) has become a promising paradigm for reasoning-oriented post-training of large language models (LLMs), especially when combined with reinforcement learn…
The Evolution of Thought: Tracking LLM Overthinking via Reasoning Dynamics Analysis
Zihao Wei, Liang Pang, Jiahao Liu +7
Test-time scaling via explicit reasoning trajectories significantly boosts large language model (LLM) performance but often triggers overthinking. To explore this, we analyze reaso…
IIET: Efficient Numerical Transformer via Implicit Iterative Euler Method
Xinyu Liu, Bei Li, Jiahao Liu +6
High-order numerical methods enhance Transformer performance in tasks like NLP and CV, but introduce a performance-efficiency trade-off due to increased computational overhead. Our…
TCPO: Thought-Centric Preference Optimization for Effective Embodied Decision-making
Kechen Jiao, Zhirui Fang, Jiahao Liu +9
Using effective generalization capabilities of vision language models (VLMs) in context-specific dynamic tasks for embodied artificial intelligence remains a significant challenge.…
Libra: Assessing and Improving Reward Model by Learning to Think
Meng Zhou, Bei Li, Jiahao Liu +5
Reinforcement learning (RL) has significantly improved the reasoning ability of large language models. However, current reward models underperform in challenging reasoning scenario…