10 papers
DeepTool: Scaling Interleaved Deliberation in Tool-Integrated Reasoning via Process-Supervised Reinforcement Learning
Yang He, Xiao Ding, Bibo Cai +5
Tool-Integrated Reasoning (TIR) extends LLM capabilities by leveraging external environments. However, existing methods lack the deliberation during sequential tool invocation requ…
GR-Ben: A General Reasoning Benchmark for Evaluating Process Reward Models
Zhouhao Sun, Xuan Zhang, Xiao Ding +10
Currently, process reward models (PRMs) have exhibited remarkable potential for test-time scaling. Since large language models (LLMs) regularly generate flawed intermediate reasoni…
TinyJudge: Unverifiable Constraint Alignment via Lightweight Specialist Ensembles
Yirong Zeng, Yufei Liu, Xiao Ding +9
Instruction Following (IF) is a core capability of LLMs, requiring strict adherence to diverse constraints, ranging from verifiable ones (e.g., output length) to unverifiable ones…
Large Language Models Are Still Misled by Simple Bias Ensembles
Zhouhao Sun, Zhiyuan Kan, Xiao Ding +5
With the evolution of large language models (LLMs), their robustness against individual simple biases has been enhanced. However, we observe that the ensemble of multiple simple bi…
Consolidation or Adaptation? PRISM: Disentangling SFT and RL Data via Gradient Concentration
Yang Zhao, Yangou Ouyang, Xiao Ding +8
While Hybrid Supervised Fine-Tuning (SFT) followed by Reinforcement Learning (RL) has become the standard paradigm for training LLM agents, effective mechanisms for data allocation…
MAESTRO: Meta-learning Adaptive Estimation of Scalarization Trade-offs for Reward Optimization
Yang Zhao, Hepeng Wang, Xiao Ding +8
Group-Relative Policy Optimization (GRPO) has emerged as an efficient paradigm for aligning Large Language Models (LLMs), yet its efficacy is primarily confined to domains with ver…