5 papers
BLADE: Boundary-Expanded and Layer-Adaptive Dynamic Exit for Efficient LLM Reasoning
Keshu Fu, Keqin Peng, Jun Bai +6
Large language models often improve task performance by generating long reasoning traces, but the resulting computation is frequently wasted on redundant verification and revision.…
Diagnosing and Mitigating Thinking Collapse in On-Policy Self-Distillation
Keqin Peng, Chen Li, Yuanxin Ouyang +2
On-Policy Self-Distillation (OPSD) has emerged as a crucial paradigm for enhancing and aligning Large Language Models (LLMs). However, in complex reasoning tasks, OPSD paradoxicall…
Think Dense, Not Long: Dynamic Decoupled Conditional Advantage for Efficient Reasoning
Keqin Peng, Yuanxin Ouyang, Xuebo Liu +4
Reinforcement Learning with Verifiable Rewards (RLVR) can elicit strong multi-step reasoning, yet it often encourages overly verbose traces. Moreover, naive length penalties in gro…
Enhancing Input-Label Mapping in In-Context Learning with Contrastive Decoding
Keqin Peng, Liang Ding, Yuanxin Ouyang +3
Large language models (LLMs) excel at a range of tasks through in-context learning (ICL), where only a few task examples guide their predictions. However, prior research highlights…
Revisiting Overthinking in Long Chain-of-Thought from the Perspective of Self-Doubt
Keqin Peng, Liang Ding, Yuanxin Ouyang +2
Reasoning Large Language Models (RLLMs) have demonstrated impressive performance on complex tasks, largely due to the adoption of Long Chain-of-Thought (Long CoT) reasoning. Howeve…