Defense-Prefix for Preventing Typographic Attacks on CLIP

Hiroki Azuma, Yusuke Matsui

Vision-language pre-training models (VLPs) have exhibited revolutionary improvements in various vision-language tasks. In VLP, some adversarial attacks fool a model into false or a…