8 papers
More Bang for the Buck: Process Reward Modeling with Entropy-Driven Uncertainty
Lang Cao, Renhong Chen, Yingtian Zou +9
We introduce the Entropy-Driven Uncertainty Process Reward Model (EDU-PRM), a novel entropy-driven training framework for process reward modeling that enables dynamic, uncertainty-…
AutoTool: Automatic Scaling of Tool-Use Capabilities in RL via Decoupled Entropy Constraints
Yirong Zeng, Xiao Ding, Yufei Liu +9
Tool use represents a critical capability for AI agents, with recent advances focusing on leveraging reinforcement learning (RL) to scale up the explicit reasoning process to achie…
The Tool-Overuse Illusion: Why Does LLM Prefer External Tools over Internal Knowledge?
Yirong Zeng, Shen You, Yufei Liu +9
Equipping LLMs with external tools effectively addresses internal reasoning limitations. However, it introduces a critical yet under-explored phenomenon: tool overuse, the unnecess…
Precision over Diversity: High-Precision Reward Generalizes to Robust Instruction Following
Yirong Zeng, Yufei Liu, Xiao Ding +9
A central belief in scaling reinforcement learning with verifiable rewards for instruction following (IF) tasks is that, a diverse mixture of verifiable hard and unverifiable soft…
Tool Zero: Training Tool-Augmented LLMs via Pure RL from Scratch
Yirong Zeng, Xiao Ding, Yutai Hou +9
Training tool-augmented LLMs has emerged as a promising approach to enhancing language models' capabilities for complex tasks. The current supervised fine-tuning paradigm relies on…
iTool: Reinforced Fine-Tuning with Dynamic Deficiency Calibration for Advanced Tool Use
Yirong Zeng, Xiao Ding, Yuxian Wang +8
Augmenting large language models (LLMs) with external tools is a promising approach to enhance their capabilities, especially for complex tasks. Synthesizing tool-use data through…