5 citations · 10 across the 19 of their papers we have counts for
19 papers
Rethinking MCP Security: A Large-Scale Study of Runtime MCP Servers and Security Scanner Reliability
Pei Chen, Baichao An, Mengying Wu +6
The Model Context Protocol (MCP) has rapidly established itself as a standard interface for enabling LLM-based agents to interact with external tools and services. As MCP servers a…
The Emergence of Autonomous Penetration Capabilities in Large Language Model-Powered AI Systems
Jiaqi Luo, Jiarun Dai, Zhile Chen +8
Nowadays, the autonomous execution of cyberattacks capable of causing substantial real-world harm is widely regarded as one of the critical red lines that frontier AI systems must…
AgentCyberRange: Benchmarking Frontier AI Systems in Realistic Cyber Ranges
Fengyu Liu, Jiarun Dai, Yihe Fan +11
Frontier AI systems are increasingly capable of cybersecurity tasks, including codebase inspection, vulnerability detection, and exploitation. However, evaluating their offensive c…
CyberEvolver: Structured Self-Evolution for Cybersecurity Agents On the Fly
Yihe Fan, Changyi Li, Lichen Xu +4
LLM-based agents are increasingly used for cybersecurity tasks, but most existing systems rely on fixed, human-designed scaffolds that struggle to adapt across diverse targets and…
Invisible Threats from Model Context Protocol: Generating Stealthy Injection Payload via Tree-based Adaptive Search
Yulin Shen, Xudong Pan, Geng Hong +1
Recent advances in the Model Context Protocol (MCP) have enabled large language models (LLMs) to invoke external tools with unprecedented ease. This creates a new class of powerful…
AutoControl Arena: Synthesizing Executable Test Environments for Frontier AI Risk Evaluation
Changyi Li, Pengfei Lu, Xudong Pan +2
As Large Language Models (LLMs) evolve into autonomous agents, existing safety evaluations face a fundamental trade-off: manual benchmarks are costly, while LLM-based simulators ar…
CoTJudger: A Graph-Driven Framework for Automatic Evaluation of Chain-of-Thought Efficiency and Redundancy in LRMs
Siyi Li, Jiajun Shi, Shiwen Ni +9
Large Reasoning Models (LRMs) have demonstrated strong performance by producing extended Chain-of-Thought (CoT) traces before answering. However, this paradigm often induces over-r…
WebTrap Park: An Automated Platform for Systematic Security Evaluation of Web Agents
Xinyi Wu, Jiagui Chen, Geng Hong +4
Web Agents are increasingly deployed to perform complex tasks in real web environments, yet their security evaluation remains fragmented and difficult to standardize. We present We…
MCPZoo: A Large-Scale Dataset of Runnable Model Context Protocol Servers for AI Agent
Mengying Wu, Pei Chen, Geng Hong +6
Model Context Protocol (MCP) enables agents to interact with external tools, yet empirical research on MCP is hindered by the lack of large-scale, accessible datasets. We present M…
Shell or Nothing: Real-World Benchmarks and Memory-Activated Agents for Automated Penetration Testing
Wuyuao Mai, Geng Hong, Qi Liu +5
Penetration testing is critical for identifying and mitigating security vulnerabilities, yet traditional approaches remain expensive, time-consuming, and dependent on expert human…
PRISON: Unmasking the Criminal Potential of Large Language Models
Xinyi Wu, Geng Hong, Pei Chen +3
As large language models (LLMs) advance, concerns about their misconduct in complex social contexts intensify. Existing research overlooked the systematic understanding and assessm…
Evaluation Faking: Unveiling Observer Effects in Safety Evaluation of Frontier AI Systems
Yihe Fan, Wenqi Zhang, Xudong Pan +1
As foundation models grow increasingly more intelligent, reliable and trustworthy safety evaluation becomes more indispensable than ever. However, an important question arises: Whe…
ReasoningShield: Safety Detection over Reasoning Traces of Large Reasoning Models
Changyi Li, Jiayi Wang, Xudong Pan +2
Large Reasoning Models (LRMs) leverage explicit reasoning traces, known as Chain-of-Thought (CoT) reasoning, to decompose complex problems into intermediate steps before deriving f…
Think Twice Before You Act: Enhancing Agent Behavioral Safety with Thought Correction
Changyue Jiang, Wenqi Zhang, Xudong Pan +2
LLM-based agents solve complex tasks through iterative reasoning, tool use, and environment interaction, where each intermediate thought directly shapes subsequent actions. Small d…
Bare Minimum Mitigations for Autonomous AI Development
Joshua Clymer, Isabella Duan, Chris Cundy +10
Artificial intelligence (AI) is advancing rapidly, with the potential for significantly automating AI research and development itself in the near future. In 2024, international sci…
OpenDeception: Learning Deception and Trust in Human-AI Interaction via Multi-Agent Simulation
Yichen Wu, Qianqian Gao, Xudong Pan +2
As large language models (LLMs) are increasingly deployed as interactive agents, open-ended human-AI interactions can involve deceptive behaviors with serious real-world consequenc…
Large language model-powered AI systems achieve self-replication with no human intervention
Xudong Pan, Jiarun Dai, Yihe Fan +3
Self-replication with no human intervention is broadly recognized as one of the principal red lines associated with frontier AI systems. While leading corporations such as OpenAI a…
Frontier AI systems have surpassed the self-replicating red line
Xudong Pan, Jiarun Dai, Yihe Fan +1
Successful self-replication under no human assistance is the essential step for AI to outsmart the human beings, and is an early signal for rogue AIs. That is why self-replication…
Feedback-Guided Extraction of Knowledge Base from Retrieval-Augmented LLM Applications
Changyue Jiang, Xudong Pan, Geng Hong +3
Retrieval-Augmented Generation (RAG) expands the knowledge boundary of large language models (LLMs) by integrating external knowledge bases, whose construction is often time-consum…