9 papers · 1 filter
Towards Scalable RLVR: Multimodal Instruction Following Data Synthesis and Distillation
Yirong Zeng, Zhang Sai, Yuxian Wang +4
Multimodal instruction following (MMIF) is crucial for building generalist agents. However, current training paradigms rely heavily on Supervised Fine-Tuning (SFT), which often lea…
EnvCraft: Synthesizing Executable Environments in Agentic RL for Claw-like Agent
Yirong Zeng, Shen You, Jinhang Feng +8
The paradigm of LLMs has rapidly shifted from passive language interfaces to autonomous Claw-like agents that execute long-horizon tasks across stateful workspaces. While Agentic R…
TinyJudge: Unverifiable Constraint Alignment via Lightweight Specialist Ensembles
Yirong Zeng, Yufei Liu, Xiao Ding +9
Instruction Following (IF) is a core capability of LLMs, requiring strict adherence to diverse constraints, ranging from verifiable ones (e.g., output length) to unverifiable ones…
DeepTool: Scaling Interleaved Deliberation in Tool-Integrated Reasoning via Process-Supervised Reinforcement Learning
Yang He, Xiao Ding, Bibo Cai +5
Tool-Integrated Reasoning (TIR) extends LLM capabilities by leveraging external environments. However, existing methods lack the deliberation during sequential tool invocation requ…
LSR-Ben: A Logical and Scientific Reasoning Benchmark for Evaluating Process Reward Models
Zhouhao Sun, Xuan Zhang, Xiao Ding +10
Currently, process reward models (PRMs) have exhibited remarkable potential for test-time scaling. Since large language models (LLMs) regularly generate flawed intermediate reasoni…
The Tool-Overuse Illusion: Why Does LLM Prefer External Tools over Internal Knowledge?
Yirong Zeng, Shen You, Yufei Liu +9
Equipping LLMs with external tools effectively addresses internal reasoning limitations. However, it introduces a critical yet under-explored phenomenon: tool overuse, the unnecess…
Precision over Diversity: High-Precision Reward Generalizes to Robust Instruction Following
Yirong Zeng, Yufei Liu, Xiao Ding +9
A central belief in scaling reinforcement learning with verifiable rewards for instruction following (IF) tasks is that, a diverse mixture of verifiable hard and unverifiable soft…
Consolidation or Adaptation? PRISM: Disentangling SFT and RL Data via Gradient Concentration
Yang Zhao, Yangou Ouyang, Xiao Ding +8
While Hybrid Supervised Fine-Tuning (SFT) followed by Reinforcement Learning (RL) has become the standard paradigm for training LLM agents, effective mechanisms for data allocation…
MAESTRO: Meta-learning Adaptive Estimation of Scalarization Trade-offs for Reward Optimization
Yang Zhao, Hepeng Wang, Xiao Ding +8
Group-Relative Policy Optimization (GRPO) has emerged as an efficient paradigm for aligning Large Language Models (LLMs), yet its efficacy is primarily confined to domains with ver…