Localize and Neutralize: Gradient-guided Token Suppression against Visual Prompt Injection Attack

Dongpeng Zhang, Ke Ma, Yangbangyan Jiang +4

Adversarial images pose a severe security threat to multimodal large language models through prompt injection. Existing defenses largely lack a principled understanding of the unde…