17 citations · 27 across the 13 of their papers we have counts for
13 papers
Repeat-After-Me: Black-Box Adaptive Visual Prompt Injection
Sizhe Chen, Yu-Lin Tsai, Ivan Evtimov +4
Prompt injection is widely recognized as a major security threat to AI agents that interact with untrusted external data, such as websites, documents, and emails. Prior work has sh…
SecOPD: Mitigating Adaptive Prompt Injections by On-Policy Distillation
Yibo Peng, Long Lian, David Wagner +1
Prompt injection is listed as the \#1 threat to AI agents. When an agent accesses external data from websites, files, or emails, an attacker may inject a prompt into the data, sayi…
Reference-Based Distillation Detection in LLMs
Rajat Rawat, Sizhe Chen, Akshay Anand +3
Model distillation -- training on outputs from stronger third-party models -- is widely used to boost performance, but raises concerns about unfair advantages and policy violations…
Defending Against Prompt Injection with DataFilter
Yizhu Wang, Sizhe Chen, Raghad Alkhudair +2
When large language model (LLM) agents are increasingly deployed to automate tasks and interact with untrusted external data, prompt injection emerges as a significant security thr…
Defending Against Prompt Injection With a Few DefensiveTokens
Sizhe Chen, Yizhu Wang, Nicholas Carlini +2
When large language model (LLM) systems interact with external data to perform complex tasks, a new attack, namely prompt injection, becomes a significant threat. By injecting inst…
Meta-SecAlign: Training LLMs against Prompt Injection for Robust Agents
Sizhe Chen, Arman Zharmagambetov, David Wagner +1
Prompt injection attacks, where untrusted data contains an injected prompt to manipulate the system, have been listed as the top security threat to AI agents. By fine-tuning on sim…
SecAlign: Defending Against Prompt Injection with Preference Optimization
Sizhe Chen, Arman Zharmagambetov, Saeed Mahloujifar +3
Large language models (LLMs) are becoming increasingly prevalent in modern software systems, interfacing between the user and the Internet to assist with tasks that require advance…
StruQ: Defending Against Prompt Injection with Structured Queries
Sizhe Chen, Julien Piet, Chawin Sitawarin +1
Recent advances in Large Language Models (LLMs) enable exciting LLM-integrated applications, which perform text-based tasks by utilizing their advanced language understanding capab…
Jatmo: Prompt Injection Defense by Task-Specific Finetuning
Julien Piet, Maha Alrashed, Chawin Sitawarin +5
Large Language Models (LLMs) are attracting significant research attention due to their instruction-following abilities, allowing users and developers to leverage LLMs for a variet…
Can LLMs Follow Simple Rules?
Norman Mu, Sarah Chen, Zifan Wang +6
As Large Language Models (LLMs) are deployed with increasing real-world responsibilities, it is important to be able to specify and constrain the behavior of these systems in a rel…
Subspace Adversarial Training
Tao Li, Yingwen Wu, Sizhe Chen +2
Single-step adversarial training (AT) has received wide attention as it proved to be both efficient and robust. However, a serious problem of catastrophic overfitting exists, i.e.,…
Measuring the Transferability of Attacks by the Norm
Sizhe Chen, Qinghua Tao, Zhixing Ye +1
Deep neural networks could be fooled by adversarial examples with trivial differences to original samples. To keep the difference imperceptible in human eyes, researchers bound the…
Relevance Attack on Detectors
Sizhe Chen, Fan He, Xiaolin Huang +1
This paper focuses on high-transferable adversarial attacks on detectors, which are hard to attack in a black-box manner, because of their multiple-output characteristics and the d…