3 citations · 4 across the 11 of their papers we have counts for
13 papers
CUE-Mem: Benchmarking Long-Term User Memory via Implicit Cues in Multimodal Conversations
Yulin Hu, Yanyan Zhao, Zimo Long +6
Long-term memory is essential for multimodal agents that interact with users across sustained conversations. However, user memories are not always explicitly stated: they may also…
Towards Scalable RLVR: Multimodal Instruction Following Data Synthesis and Distillation
Yirong Zeng, Zhang Sai, Yuxian Wang +4
Multimodal instruction following (MMIF) is crucial for building generalist agents. However, current training paradigms rely heavily on Supervised Fine-Tuning (SFT), which often lea…
EnvCraft: Synthesizing Executable Environments in Agentic RL for Claw-like Agent
Yirong Zeng, Shen You, Jinhang Feng +8
The paradigm of LLMs has rapidly shifted from passive language interfaces to autonomous Claw-like agents that execute long-horizon tasks across stateful workspaces. While Agentic R…
TinyJudge: Unverifiable Constraint Alignment via Lightweight Specialist Ensembles
Yirong Zeng, Yufei Liu, Xiao Ding +9
Instruction Following (IF) is a core capability of LLMs, requiring strict adherence to diverse constraints, ranging from verifiable ones (e.g., output length) to unverifiable ones…
Safety Geometry Collapse in Multimodal LLMs and Adaptive Drift Correction
Jiahe Guo, Xiangran Guo, Jiaxuan Chen +6
Multimodal large language models (MLLMs) often fail to transfer safety capabilities learned in the text modality to semantically equivalent non-text inputs, revealing a persistent…
AutoTool: Automatic Scaling of Tool-Use Capabilities in RL via Decoupled Entropy Constraints
Yirong Zeng, Xiao Ding, Yufei Liu +9
Tool use represents a critical capability for AI agents, with recent advances focusing on leveraging reinforcement learning (RL) to scale up the explicit reasoning process to achie…
The Tool-Overuse Illusion: Why Does LLM Prefer External Tools over Internal Knowledge?
Yirong Zeng, Shen You, Yufei Liu +9
Equipping LLMs with external tools effectively addresses internal reasoning limitations. However, it introduces a critical yet under-explored phenomenon: tool overuse, the unnecess…
Precision over Diversity: High-Precision Reward Generalizes to Robust Instruction Following
Yirong Zeng, Yufei Liu, Xiao Ding +9
A central belief in scaling reinforcement learning with verifiable rewards for instruction following (IF) tasks is that, a diverse mixture of verifiable hard and unverifiable soft…
Tool Zero: Training Tool-Augmented LLMs via Pure RL from Scratch
Yirong Zeng, Xiao Ding, Yutai Hou +9
Training tool-augmented LLMs has emerged as a promising approach to enhancing language models' capabilities for complex tasks. The current supervised fine-tuning paradigm relies on…
iTool: Reinforced Fine-Tuning with Dynamic Deficiency Calibration for Advanced Tool Use
Yirong Zeng, Xiao Ding, Yuxian Wang +8
Augmenting large language models (LLMs) with external tools is a promising approach to enhance their capabilities, especially for complex tasks. Synthesizing tool-use data through…
ToolACE: Winning the Points of LLM Function Calling
Weiwen Liu, Xu Huang, Xingshan Zeng +24
Function calling significantly extends the application boundary of large language models, where high-quality and diverse training data is critical for unlocking this capability. Ho…
Concise and Precise Context Compression for Tool-Using Language Models
Yang Xu, Yunlong Feng, Honglin Mu +9
Through reading the documentation in the context, tool-using language models can dynamically extend their capability using external tools. The cost is that we have to input lengthy…
Planning, Creation, Usage: Benchmarking LLMs for Comprehensive Tool Utilization in Real-World Complex Scenarios
Shijue Huang, Wanjun Zhong, Jianqiao Lu +10
The recent trend of using Large Language Models (LLMs) as tool agents in real-world applications underscores the necessity for comprehensive evaluations of their capabilities, part…