Security--Fidelity Tradeoffs: The Hidden Cost of Prompt Injection Defense

Mitchell Hermon, Rahul Gupta, Weitong Ruan +2

We identify a security-fidelity tradeoff in defending LLMs against indirect prompt injection: defenses resist injected instructions largely by suppressing untrusted text, which cor…