6 papers · 1 filter
BackdoorVLM: A Benchmark for Backdoor Attacks on Vision-Language Models
Juncheng Li, Yige Li, Hanxun Huang +5
Backdoor attacks undermine the reliability and trustworthiness of machine learning systems by injecting hidden behaviors that can be maliciously activated at inference time. While…
Simulated Ensemble Attack: Transferring Jailbreaks Across Fine-tuned Vision-Language Models
Ruofan Wang, Xin Wang, Yang Yao +3
The widespread practice of fine-tuning open-source Vision-Language Models (VLMs) raises a critical security concern: jailbreak vulnerabilities in base models may persist in downstr…
NAP-Tuning: Neural Augmented Prompt Tuning for Adversarially Robust Vision-Language Models
Jiaming Zhang, Xin Wang, Xingjun Ma +3
Vision-Language Models (VLMs) such as CLIP have demonstrated remarkable capabilities in understanding relationships between visual and textual data through joint embedding spaces.…
X-Transfer Attacks: Towards Super Transferable Adversarial Attacks on CLIP
Hanxun Huang, Sarah Erfani, Yige Li +2
As Contrastive Language-Image Pre-training (CLIP) models are increasingly adopted for diverse downstream tasks and integrated into large vision-language models (VLMs), their suscep…
TAPT: Test-Time Adversarial Prompt Tuning for Robust Inference in Vision-Language Models
Xin Wang, Kai Chen, Jiaming Zhang +2
Large pre-trained Vision-Language Models (VLMs) such as CLIP have demonstrated excellent zero-shot generalizability across various downstream tasks. However, recent studies have sh…
Adversarial Prompt Distillation for Vision-Language Models
Lin Luo, Xin Wang, Bojia Zi +3
Large pre-trained Vision-Language Models (VLMs) such as Contrastive Language-Image Pre-training (CLIP) have been shown to be susceptible to adversarial attacks, raising concerns ab…