6 papers
TAME: Test-Time Adversarial Prompt Tuning via Mixture-of-Experts for Vision-Language Models
Xin Wang, Yixu Wang, Jiaming Zhang +6
Large-scale pre-trained Vision-Language models (VLMs), such as CLIP, exhibit strong zero-shot generalization, yet remain highly vulnerable to imperceptible adversarial perturbation…
RedDiffuser: Auditing Multimodal Safety Failures in Vision-Language Models via Reinforced Diffusion
Ruofan Wang, Xingjun Ma
Large Vision-Language Models (VLMs) are increasingly deployed in open-ended environments, where ensuring reliable safety under multimodal inputs is critical. However, existing eval…
Safety at Scale: A Comprehensive Survey of Large Model and Agent Safety
Xingjun Ma, Yifeng Gao, Yixu Wang +45
The rapid advancement of large models, driven by their exceptional abilities in learning and generalization through large-scale pre-training, has reshaped the landscape of Artifici…
Simulated Ensemble Attack: Transferring Jailbreaks Across Fine-tuned Vision-Language Models
Ruofan Wang, Xin Wang, Yang Yao +3
The widespread practice of fine-tuning open-source Vision-Language Models (VLMs) raises a critical security concern: jailbreak vulnerabilities in base models may persist in downstr…
IDEATOR: Jailbreaking and Benchmarking Large Vision-Language Models Using Themselves
Ruofan Wang, Juncheng Li, Yixu Wang +6
As large Vision-Language Models (VLMs) gain prominence, ensuring their safe deployment has become critical. Recent studies have explored VLM robustness against jailbreak attacks-te…
White-box Multimodal Jailbreaks Against Large Vision-Language Models
Ruofan Wang, Xingjun Ma, Hanxu Zhou +3
Recent advancements in Large Vision-Language Models (VLMs) have underscored their superiority in various multimodal tasks. However, the adversarial robustness of VLMs has not been…