4 papers · 1 filter
Test-Time Attention Purification for Backdoored Large Vision Language Models
Zhifang Zhang, Bojun Yang, Shuo He +5
Despite the strong multimodal performance, large vision-language models (LVLMs) are vulnerable during fine-tuning to backdoor attacks, where adversaries insert trigger-embedded sam…
Defending Multimodal Backdoored Models by Repulsive Visual Prompt Tuning
Zhifang Zhang, Shuo He, Haobo Wang +2
Multimodal contrastive learning models (e.g., CLIP) can learn high-quality representations from large-scale image-text datasets, while they exhibit significant vulnerabilities to b…
Improving Generalizability and Undetectability for Targeted Adversarial Attacks on Multimodal Pre-trained Models
Zhifang Zhang, Jiahan Zhang, Shengjie Zhou +4
Multimodal pre-trained models (e.g., ImageBind), which align distinct data modalities into a shared embedding space, have shown remarkable success across downstream tasks. However,…
Test-Time Multimodal Backdoor Detection by Contrastive Prompting
Yuwei Niu, Shuo He, Qi Wei +3
While multimodal contrastive learning methods (e.g., CLIP) can achieve impressive zero-shot classification performance, recent research has revealed that these methods are vulnerab…