14 citations · 15 across the 15 of their papers we have counts for
15 papers · 1 filter
Black-Box Red-Teaming of Multi-Agent Systems via Reinforcement Learning
Yuqi Jia, Hangsheng Zhang, Chenxi Liu +2
Large language model (LLM) agents are increasingly deployed in multi-agent systems (MAS) to accomplish complex tasks. Prior black-box red-teaming attacks mainly focus on single age…
LongPIBench: A Long-Context Benchmark for Prompt Injection
Yupei Liu, Yuqi Jia, Neil Zhenqiang Gong +1
Prompt injection attacks pose a serious security risk to large language models in real-world applications. However, existing prompt injection benchmarks primarily focus on short-co…
ContextLeak: Exfiltrating LLM Agent Context via Malicious Tools
Yuqi Jia, Ruiqi Wang, Patrick Li +3
Exfiltrating an LLM agent's runtime context -- such as the user prompt, execution trajectory, and tool list -- poses severe security and privacy risks to users. Such attacks can be…
Measuring Real-World Prompt Injection Attacks in LLM-based Resume Screening
Mohan Zhang, Yuqi Jia, Zhen Tan +4
LLMs are vulnerable to prompt injection attacks. However, this vulnerability has been primarily demonstrated conceptually in academic studies or through a few anecdotal case studie…
AlignSentinel: Alignment-Aware Detection of Prompt Injection Attacks
Yuqi Jia, Ruiqi Wang, Xilong Wang +2
Prompt injection attacks insert malicious instructions into an LLM's input to steer it toward an attacker-chosen task instead of the intended one. Existing detection defenses typic…
MalTool: Malicious Tool Attacks on LLM Agents
Yuepeng Hu, Yuqi Jia, Mengyuan Li +2
In a malicious tool attack, an attacker uploads a malicious tool to a distribution platform; once a user inadvertently installs the tool and the LLM agent selects it during task ex…
ObliInjection: Order-Oblivious Prompt Injection Attack to LLM Agents with Multi-source Data
Reachal Wang, Yuqi Jia, Neil Zhenqiang Gong
Prompt injection attacks aim to contaminate the input data of an LLM to mislead it into completing an attacker-chosen task instead of the intended task. In many applications and ag…
PromptLocate: Localizing Prompt Injection Attacks
Yuqi Jia, Yupei Liu, Zedian Shao +2
Prompt injection attacks deceive a large language model into completing an attacker-specified task instead of its intended task by contaminating its input data with an injected pro…
WAInjectBench: Benchmarking Prompt Injection Detections for Web Agents
Yinuo Liu, Ruohan Xu, Xilong Wang +2
Multiple prompt injection attacks have been proposed against web agents. At the same time, various methods have been developed to detect general prompt injection attacks, but none…
SecInfer: Preventing Prompt Injection via Inference-time Scaling
Yupei Liu, Yanting Wang, Yuqi Jia +2
Prompt injection attacks pose a pervasive threat to the security of Large Language Models (LLMs). State-of-the-art prevention-based defenses typically rely on fine-tuning an LLM to…
PromptArmor: Simple yet Effective Prompt Injection Defenses
Tianneng Shi, Kaijie Zhu, Zhun Wang +13
Despite their potential, recent research has demonstrated that LLM agents are vulnerable to prompt injection attacks, where malicious prompts are injected into the agent's input, c…
A Critical Evaluation of Defenses against Prompt Injection Attacks
Yuqi Jia, Zedian Shao, Yupei Liu +3
Large Language Models (LLMs) are vulnerable to prompt injection attacks, and several defenses have recently been proposed, often claiming to mitigate these attacks successfully. Ho…
DataSentinel: A Game-Theoretic Detection of Prompt Injection Attacks
Yupei Liu, Yuqi Jia, Jinyuan Jia +2
LLM-integrated applications and agents are vulnerable to prompt injection attacks, where an attacker injects prompts into their inputs to induce attacker-desired outputs. A detecti…
Evaluating LLM-based Personal Information Extraction and Countermeasures
Yupei Liu, Yuqi Jia, Jinyuan Jia +1
Automatically extracting personal information -- such as name, phone number, and email address -- from publicly available profiles at a large scale is a stepstone to many other sec…
Formalizing and Benchmarking Prompt Injection Attacks and Defenses
Yupei Liu, Yuqi Jia, Runpeng Geng +2
A prompt injection attack aims to inject malicious instruction/data into the input of an LLM-Integrated Application such that it produces results as an attacker desires. Existing w…