most citedYour Agent May Misevolve: Emergent Risks in Self-evolving LLM Agents

1 citations · 1 across the 15 of their papers we have counts for

collaborators
Showing cs.AIShow all

13 papers · 1 filter

cs.AI2026

StepGuard: Learning Step-Level Guardrails with Scalable Supervision and Safety-Utility Balancing

Zhijie Zheng, Yu Li, Chen Qian +5

LLM-based agents can interact with external environments through tool invocation, but this capability also introduces security risks such as file modification, information leakage,…

cs.AI2026

SHE: Trajectory-driven Safety Harness Evolution for LLM Agents

Wanying Qu, Qinghua Mao, Yu Li +12

The safety of large language model (LLM) agents depends not only on model weights but also on the agent harness that manages context, memory, tools, permissions, and runtime contro…

cs.AI2026

Exploring Agentic Tool-Calling Decisions via Uncertainty-Aligned Reinforcement Learning

Yijin Zhou, Linqian Zeng, Xiaoya Lu +4

Large language model (LLM)-based agents often make suboptimal tool-use decisions, including unsupported tool invocation and hallucinated direct responses, which may accumulate erro…

cs.AI2026

COLLEAGUE.SKILL: Automated AI Skill Generation via Expert Knowledge Distillation

Tianyi Zhou, Dongrui Liu, Leitao Yuan +2

LLM agents are increasingly expected not only to complete isolated tasks, but also to carry bounded representations of human expertise, judgment, and interaction style. Building su…

cs.AI2026

Safactory: A Scalable Agentic Infrastructure for Training Trustworthy Autonomous Intelligence

Xinquan Chen, Zhenyun Yin, Shan He +38

As large models evolve from conversational assistants into autonomous agents, challenges increasingly arise from long-horizon decision making, tool use, and real environment intera…

cs.AI2026

Benchmarks for Trajectory Safety Evaluation and Diagnosis in OpenClaw and Codex: ATBench-Claw and ATBench-Codex

Zhonghao Yang, Yu Li, Yanxu Zhu +6

As agent systems move into increasingly diverse execution settings, trajectory-level safety evaluation and diagnosis require benchmarks that evolve with them. ATBench is a diverse…

cs.AI2026

Rethinking Generalization in Reasoning SFT: A Conditional Analysis on Optimization, Data, and Model Capability

Qihan Ren, Peng Wang, Ruikun Cai +8

A prevailing narrative in LLM post-training holds that supervised finetuning (SFT) memorizes while reinforcement learning (RL) generalizes. We revisit this claim for reasoning SFT…

cs.AI2026

ATBench: A Diverse and Realistic Agent Trajectory Benchmark for Safety Evaluation and Diagnosis

Yu Li, Haoyu Luo, Yuejin Xie +10

Evaluating the safety of LLM-based agents is increasingly important because risks in realistic deployments often emerge over multi-step interactions rather than isolated prompts or…

cs.AI2026

The Why Behind the Action: Unveiling Internal Drivers via Agentic Attribution

Chen Qian, Peng Wang, Dongrui Liu +10

Large Language Model (LLM)-based agents are widely used in real-world applications such as customer service, web navigation, and software engineering. As these systems become more…

cs.AI2026

AgentDoG: A Diagnostic Guardrail Framework for AI Agent Safety and Security

Dongrui Liu, Qihan Ren, Chen Qian +40

The rise of AI agents introduces complex safety and security challenges arising from autonomous tool use and environmental interactions. Current guardrail models lack agentic risk…

cs.AI2025

Are Your Agents Upward Deceivers?

Dadi Guo, Qingyu Liu, Dongrui Liu +13

Large Language Model (LLM)-based agents are increasingly used as autonomous subordinates that carry out tasks for users. This raises the question of whether they may also engage in…

cs.AI2025

Towards Self-Evolving Benchmarks: Synthesizing Agent Trajectories via Test-Time Exploration under Validate-by-Reproduce Paradigm

Dadi Guo, Tianyi Zhou, Dongrui Liu +8

Recent advances in large language models (LLMs) and agent system designs have empowered agents with unprecedented levels of capability. However, existing agent benchmarks are showi…

cs.AI2025★ 1 cited

Your Agent May Misevolve: Emergent Risks in Self-evolving LLM Agents

Shuai Shao, Qihan Ren, Chen Qian +8

Advances in Large Language Models (LLMs) have enabled a new class of self-evolving agents that autonomously improve through interaction with the environment, demonstrating strong c…