6 citations · 19 across the 14 of their papers we have counts for
14 papers
Repeat-After-Me: Black-Box Adaptive Visual Prompt Injection
Sizhe Chen, Yu-Lin Tsai, Ivan Evtimov +4
Prompt injection is widely recognized as a major security threat to AI agents that interact with untrusted external data, such as websites, documents, and emails. Prior work has sh…
SecOPD: Mitigating Adaptive Prompt Injections by On-Policy Distillation
Yibo Peng, Long Lian, David Wagner +1
Prompt injection is listed as the \#1 threat to AI agents. When an agent accesses external data from websites, files, or emails, an attacker may inject a prompt into the data, sayi…
SeedAIchemy: LLM-Driven Seed Corpus Generation for Fuzzing
Aidan Wen, Norah A. Alzahrani, Jingzhi Jiang +5
We introduce SeedAIchemy, an automated LLM-driven corpus generation tool that makes it easier for developers to implement fuzzing effectively. SeedAIchemy consists of five modules…
Defending Against Prompt Injection with DataFilter
Yizhu Wang, Sizhe Chen, Raghad Alkhudair +2
When large language model (LLM) agents are increasingly deployed to automate tasks and interact with untrusted external data, prompt injection emerges as a significant security thr…
Defending Against Prompt Injection With a Few DefensiveTokens
Sizhe Chen, Yizhu Wang, Nicholas Carlini +2
When large language model (LLM) systems interact with external data to perform complex tasks, a new attack, namely prompt injection, becomes a significant threat. By injecting inst…
Meta-SecAlign: Training LLMs against Prompt Injection for Robust Agents
Sizhe Chen, Arman Zharmagambetov, David Wagner +1
Prompt injection attacks, where untrusted data contains an injected prompt to manipulate the system, have been listed as the top security threat to AI agents. By fine-tuning on sim…
SCGAgent: Recreating the Benefits of Reasoning Models for Secure Code Generation with Agentic Workflows
Rebecca Saul, Hao Wang, Koushik Sen +1
Large language models (LLMs) have seen widespread success in code generation tasks for different scenarios, both everyday and professional. However current LLMs, despite producing…
Semantic-Aware Parsing for Security Logs
Julien Piet, Vivian Fang, Rishi Khare +4
Security logs are foundational to threat detection and post-incident investigation, yet analysts often struggle to fully leverage them due to their heterogeneity and unstructured n…
A Closer Look at System Prompt Robustness
Norman Mu, Jonathan Lu, Michael Lavery +1
System prompts have emerged as a critical control surface for specifying the behavior of LLMs in chat and agent settings. Developers depend on system prompts to specify important c…
Certifiably Robust RAG against Retrieval Corruption
Chong Xiang, Tong Wu, Zexuan Zhong +3
Retrieval-augmented generation (RAG) is susceptible to retrieval corruption attacks, where malicious passages injected into retrieval results can lead to inaccurate model responses…
PAL: Proxy-Guided Black-Box Attack on Large Language Models
Chawin Sitawarin, Norman Mu, David Wagner +1
Large Language Models (LLMs) have surged in popularity in recent months, but they have demonstrated concerning capabilities to generate harmful content when manipulated. While tech…
StruQ: Defending Against Prompt Injection with Structured Queries
Sizhe Chen, Julien Piet, Chawin Sitawarin +1
Recent advances in Large Language Models (LLMs) enable exciting LLM-integrated applications, which perform text-based tasks by utilizing their advanced language understanding capab…
Jatmo: Prompt Injection Defense by Task-Specific Finetuning
Julien Piet, Maha Alrashed, Chawin Sitawarin +5
Large Language Models (LLMs) are attracting significant research attention due to their instruction-following abilities, allowing users and developers to leverage LLMs for a variet…
Can LLMs Follow Simple Rules?
Norman Mu, Sarah Chen, Zifan Wang +6
As Large Language Models (LLMs) are deployed with increasing real-world responsibilities, it is important to be able to specify and constrain the behavior of these systems in a rel…