32 papers
Dual Adversarial Fine-tuning for Enhancing Robustness of Large Vision Language Model
Sibo Wang, Jie Zhang, Shiguang Shan +2
While Large Vision-Language Models (LVLMs), represented by LLaVA and GPT-4V, have demonstrated remarkable capabilities, their visual inputs remain vulnerable to adversarial attacks…
DeepBias: Adaptive In-depth Probing of Social Biases in LVLMs
Anqi Li, Jie Zhang, Zhongqi Wang +4
While Large Vision-Language Models (LVLMs) demonstrate remarkable capabilities, they remain highly susceptible to embedded social biases. Existing bias evaluation protocols predomi…
Neural Gate: Mitigating Privacy Risks in LVLMs via Neuron-Level Gradient Gating
Xiangkui Cao, Jie Zhang, Meina Kan +2
Large Vision-Language Models (LVLMs) have shown remarkable potential across a wide array of vision-language tasks, leading to their adoption in critical domains such as finance and…
Contrastive Spectral Rectification: Test-Time Defense towards Zero-shot Adversarial Robustness of CLIP
Sen Nie, Jie Zhang, Zhuo Wang +2
Vision-language models (VLMs) such as CLIP have demonstrated remarkable zero-shot generalization, yet remain highly vulnerable to adversarial examples (AEs). While test-time defens…
Plan-R1: Safe and Feasible Trajectory Planning as Language Modeling
Xiaolong Tang, Meina Kan, Shiguang Shan +1
Safe and feasible trajectory planning is critical for real-world autonomous driving systems. However, existing learning-based planners rely heavily on expert demonstrations, which…
EntropyScan: Towards Model-level Backdoor Detection in LVLMs via Visual Attention Entropy
Xuanyu Ge, Zhongqi Wang, Jie Zhang +2
Large Vision-Language Models (LVLMs) have demonstrated remarkable capabilities across various tasks, yet they remain vulnerable to backdoor attacks. Existing defense methods predom…