4 citations · 4 across the 8 of their papers we have counts for
9 papers
Controllable and Verifiable Tool-Use Data Synthesis for Agentic Reinforcement Learning
Siyuan Xu, Shiyang Li, Xin Liu +9
Existing synthetic tool-use corpora are primarily designed for offline supervised fine-tuning, yet reinforcement learning (RL) requires executable environments that support reward-…
Training LLMs for Multi-Step Tool Orchestration with Constrained Data Synthesis and Graduated Rewards
Cheng Jiayang, Xin Liu, Zhihan Zhang +8
Multi-step tool orchestration remains challenging for LLMs, as state-of-the-art models frequently fail on full sequence execution due to parameter errors. Training for these workfl…
Self-Rewarding PPO: Aligning Large Language Models with Demonstrations Only
Qingru Zhang, Liang Qiu, Ilgee Hong +11
Supervised fine-tuning (SFT) has emerged as a crucial method for aligning large language models (LLMs) with human-annotated demonstrations. However, SFT, being an off-policy approa…
Train a Unified Multimodal Data Quality Classifier with Synthetic Data
Weizhi Wang, Rongmei Lin, Shiyang Li +7
The Multimodal Large Language Models (MLLMs) are continually pre-trained on a mixture of image-text caption data and interleaved document data, while the high-quality data filterin…
Learning to Optimize Multi-Objective Alignment Through Dynamic Reward Weighting
Yining Lu, Zilong Wang, Shiyang Li +6
Prior work in multi-objective reinforcement learning typically uses linear reward scalarization with fixed weights, which provably fails to capture non-convex Pareto fronts and thu…
IHEval: Evaluating Language Models on Following the Instruction Hierarchy
Zhihan Zhang, Shiyang Li, Zixuan Zhang +11
The instruction hierarchy, which establishes a priority order from system messages to user messages, conversation history, and tool outputs, is essential for ensuring consistent an…
RNR: Teaching Large Language Models to Follow Roles and Rules
Kuan Wang, Alexander Bukharin, Haoming Jiang +9
Instruction fine-tuning (IFT) elicits instruction following capabilities and steers the behavior of large language models (LLMs) via supervised learning. However, existing models t…
Inductive or Deductive? Rethinking the Fundamental Reasoning Abilities of LLMs
Kewei Cheng, Jingfeng Yang, Haoming Jiang +9
Reasoning encompasses two typical types: deductive reasoning and inductive reasoning. Despite extensive research into the reasoning capabilities of Large Language Models (LLMs), mo…
Data Diversity Matters for Robust Instruction Tuning
Alexander Bukharin, Shiyang Li, Zhengyang Wang +6
Recent works have shown that by curating high quality and diverse instruction tuning datasets, we can significantly improve instruction-following capabilities. However, creating su…