QuISE: Defense against Typographic Attacks on VLMs via Query-Irrelevant Semantic Editing

Shubin Lu, Jiaqi Yin, Yihao Huang

Typographic attacks pose a critical threat to vision-language models (VLMs) by injecting misleading text into images and causing models to rely on adversarial textual cues rather t…