19 papers
Detecting Chain-of-Thought Hijacking in Backdoored Multimodal Large Reasoning Models
Zhifang Zhang, Qi Wei, Weitong Chen +5
Chain-of-thought (CoT) reasoning enables multimodal large reasoning models (MLRMs) to solve complex visual problems, but also exposes them to CoT hijacking: an adversary implants a…
Purifying Backdoored Large Vision-Language Models by Removing Hijacked Directions
Bojun Yang, Haochen Zhou, Zhifang Zhang +3
Large vision-language models (LVLMs) are increasingly deployed in safety-critical applications, yet they remain vulnerable to backdoor attacks. Defending against such attacks remai…
Credit Where It Matters: Dependency-Aware Policy Optimization for Terminal Agents
Yu Li, Guangfeng Cai, Long-Fei Li +5
Terminal-using agents benefit from reinforcement learning (RL) in coding, debugging, and other multi-step terminal tasks. In these tasks, later commands often depend on information…
Choosing Before Acting: Comparative Value Estimation for Long-Horizon Tool-Use Agents
Yu Li, Zheng Zhang, Xin Liu +3
Large language models (LLMs) rely on long-horizon tool invocation sequences for complex tasks, where each invocation can alter the task state and condition subsequent decisions. In…
SCA: Spatial Credit Assignment for Reinforcement Learning of GUI Agents
Shengtian Yang, Ziyu Xiong, Kaibing Yang +6
GUI agents automate tasks on digital devices by grounding language instructions in visual interfaces. Existing group-relative reinforcement learning improves GUI action prediction…
SRPO: Setwise Relative Policy Optimization for Multi-Agent Systems
Shengtian Yang, Ziyu Xiong, Yu Li +3
Multi-agent systems enable complex reasoning and tool use by coordinating agents that divide roles and refine candidate solutions. Existing methods typically update individual agen…
AgentBrew: Offline Tool-Use Agent Learning from Raw Real-World Trajectories
Zhiyi Lyu, Yewen Li, Longtao Zheng +7
LLM-based agents are increasingly deployed in real-world applications through tool-use APIs, yet training them for specific environments remains fundamentally difficult: real-world…
DE-Venus: A Data-Efficient RLVR Framework for Large Language Models
Shenzhi Yang, Guangcheng Zhu, Kai Tang +11
Reinforcement learning with verifiable rewards (RLVR) improves large language model reasoning, but its practical scaling is constrained by expensive on-policy rollouts and the cost…
When Agents Learn to Be You: Benchmarking Privacy Leakage, Impersonation Risk, and Defenses in Persona Skills
Yongli Xiang, Zhifang Zhang, Bojun Yang +4
Persona skills distill personal interaction histories into portable and executable artifacts for downstream agents. While enabling flexible personalization, this process concentrat…
PlatformBid: An Auto-Bidding Benchmark from a Unified Advertising Platform's Perspective
Shengtian Yang, Yewen Li, Peng Jiang +4
Real-time bidding is central to computational advertising, comprising three elements: Supply Side Platform (SSP) selling ad impressions, Demand Side Platform (DSP) bidding for adve…
Learning from All-Failed Groups via First-Visit Coverage in Long-Horizon Agentic Tasks
Kaibing Yang, Guangfeng Cai, Shengtian Yang +6
Group-based policy optimization has been increasingly used to train large language model (LLM) agents from sparse outcome rewards by comparing trajectories or steps within a group.…
Beyond Next-Observation Prediction: Agent-Authored World Modeling for Sequential Decision Making
Guangfeng Cai, Kaibing Yang, Shuo He +4
Recent studies on world modeling for Large Language Model (LLM) agents typically formulate the learning objective as next-observation prediction. However, this objective ties super…
Beyond Trajectory-Level Attribution: Graph-Based Credit Assignment for Agentic Reinforcement Learning
Xin Cheng, Shuo He, Lang Feng +4
Group-based reinforcement learning (RL) methods have achieved remarkable success in improving the performance of large language models (LLMs) and have been rapidly extended to agen…
Test-Time Attention Purification for Backdoored Large Vision Language Models
Zhifang Zhang, Bojun Yang, Shuo He +5
Despite the strong multimodal performance, large vision-language models (LVLMs) are vulnerable during fine-tuning to backdoor attacks, where adversaries insert trigger-embedded sam…
Towards Safer Large Reasoning Models by Promoting Safety Decision-Making before Chain-of-Thought Generation
Jianan Chen, Zhifang Zhang, Shuo He +3
Large reasoning models (LRMs) achieved remarkable performance via chain-of-thought (CoT), but recent studies showed that such enhanced reasoning capabilities are at the expense of…
Online Causal Kalman Filtering for Stable and Effective Policy Optimization
Shuo He, Lang Feng, Xin Cheng +2
Reinforcement learning for large language models suffers from high-variance token-level importance sampling (IS) ratios, which would destabilize policy optimization at scale. To im…
Hierarchy-of-Groups Policy Optimization for Long-Horizon Agentic Tasks
Shuo He, Lang Feng, Qi Wei +3
Group-based reinforcement learning (RL), such as GRPO, has advanced the capabilities of large language models on long-horizon agentic tasks. To enable more fine-grained policy upda…
Improving Generalizability and Undetectability for Targeted Adversarial Attacks on Multimodal Pre-trained Models
Zhifang Zhang, Jiahan Zhang, Shengjie Zhou +4
Multimodal pre-trained models (e.g., ImageBind), which align distinct data modalities into a shared embedding space, have shown remarkable success across downstream tasks. However,…
Defending Multimodal Backdoored Models by Repulsive Visual Prompt Tuning
Zhifang Zhang, Shuo He, Haobo Wang +2
Multimodal contrastive learning models (e.g., CLIP) can learn high-quality representations from large-scale image-text datasets, while they exhibit significant vulnerabilities to b…