5 papers · 1 filter
The Devil is in the Context: Nine Dimensions of Prompt Injection Attacks
Jiamin Chang, Kristina Nikolić, Rishi Sharma +7
Prompt injection is usually discussed and evaluated as a single, homogeneous type of attack. However, vulnerabilities reported in real-world systems show that prompt injections var…
LLMail-Inject: A Dataset from a Realistic Adaptive Prompt Injection Challenge
Sahar Abdelnabi, Aideen Fay, Ahmed Salem +22
Indirect prompt injection attacks exploit a fundamental weakness of LLMs: executing unauthorized tasks from untrusted sources. Despite the practical significance of this attack vec…
Dataset and Lessons Learned from the 2024 SaTML LLM Capture-the-Flag Competition
Edoardo Debenedetti, Javier Rando, Daniel Paleka +18
Large language model systems face important security risks from maliciously crafted messages that aim to overwrite the system's original instructions or leak private data. To study…
Get my drift? Catching LLM Task Drift with Activation Deltas
Sahar Abdelnabi, Aideen Fay, Giovanni Cherubin +3
LLMs are commonly used in retrieval-augmented applications to execute user instructions based on data from external sources. For example, modern search engines use LLMs to answer q…
Closed-Form Bounds for DP-SGD against Record-level Inference
Giovanni Cherubin, Boris Köpf, Andrew Paverd +3
Machine learning models trained with differentially-private (DP) algorithms such as DP-SGD enjoy resilience against a wide range of privacy attacks. Although it is possible to deri…