6 papers
The Art of Efficient Reasoning: Data, Reward, and Optimization
Taiqiang Wu, Zenan Xu, Bo Zhou +1
Large Language Models (LLMs) consistently benefit from scaled Chain-of-Thought (CoT) reasoning, but also suffer from heavy computational overhead. To address this issue, efficient…
Discovery and Reinforcement of Tool-Integrated Reasoning Chains via Rollout Trees
Kun Li, Zenan Xu, Junan Li +4
Tool-Integrated Reasoning has emerged as a key paradigm to augment Large Language Models (LLMs) with computational capabilities, yet integrating tool-use into long Chain-of-Thought…
Segmental Advantage Estimation: Enhancing PPO for Long-Context LLM Training
Xue Gong, Qi Yi, Ziyuan Nan +8
Training Large Language Models (LLMs) for reasoning tasks is increasingly driven by Reinforcement Learning with Verifiable Rewards (RLVR), where Proximal Policy Optimization (PPO)…
ConMax: Confidence-Maximizing Compression for Efficient Chain-of-Thought Reasoning
Minda Hu, Zexuan Qiu, Zenan Xu +3
Recent breakthroughs in Large Reasoning Models (LRMs) have demonstrated that extensive Chain-of-Thought (CoT) generation is critical for enabling intricate cognitive behaviors, suc…
Reinforcement Learning on Pre-Training Data
Siheng Li, Kejiao Li, Zenan Xu +33
The growing disparity between the exponential scaling of computational resources and the finite growth of high-quality text data now constrains conventional scaling approaches for…
Adaptive Termination for Multi-round Parallel Reasoning: An Universal Semantic Entropy-Guided Framework
Zenan Xu, Zexuan Qiu, Guanhua Huang +9
Recent advances in large language models (LLMs) have accelerated progress toward artificial general intelligence, with inference-time scaling emerging as a key technique. Contempor…