57 citations · 125 across the 36 of their papers we have counts for
41 papers
BadAction: Backdoor Attacks on Interactive Video Generation via Action-Guided Triggers
Zhihang Wu, Zhongqi Wang, Jie Zhang +3
Interactive video generation (IVG) models have achieved remarkable progress in producing controllable visual content guided by user-defined actions, yet their security vulnerabilit…
Backdoor as Probe: Test-Time Adversarial Defense for CLIP
Zhongqi Wang, Jie Zhang, Nie Sen +3
Test-time adversarial defense improves the robustness of vision-language foundation models such as CLIP without retraining. However, adversarial activation shifts are typically tre…
Dual Adversarial Fine-tuning for Enhancing Robustness of Large Vision Language Model
Sibo Wang, Jie Zhang, Shiguang Shan +2
While Large Vision-Language Models (LVLMs), represented by LLaVA and GPT-4V, have demonstrated remarkable capabilities, their visual inputs remain vulnerable to adversarial attacks…
TRACE: Structure-Aware Character Encoding for Robust and Generalizable Document Watermarking
Jiale Meng, Jie Zhang, Runyi Hu +3
We propose TRACE, a structure-aware framework leveraging diffusion models for localized character encoding to embed data. Unlike existing methods that rely on edge features or pre-…
What Makes VLMs Robust? Towards Reconciling Robustness and Accuracy in Vision-Language Models
Sen Nie, Jie Zhang, Zhongqi Wang +3
Achieving adversarial robustness in Vision-Language Models (VLMs) inevitably compromises accuracy on clean data, presenting a long-standing and challenging trade-off. In this work,…
Neural Gate: Mitigating Privacy Risks in LVLMs via Neuron-Level Gradient Gating
Xiangkui Cao, Jie Zhang, Meina Kan +2
Large Vision-Language Models (LVLMs) have shown remarkable potential across a wide array of vision-language tasks, leading to their adoption in critical domains such as finance and…
Contrastive Spectral Rectification: Test-Time Defense towards Zero-shot Adversarial Robustness of CLIP
Sen Nie, Jie Zhang, Zhuo Wang +2
Vision-language models (VLMs) such as CLIP have demonstrated remarkable zero-shot generalization, yet remain highly vulnerable to adversarial examples (AEs). While test-time defens…
Steering Vision-Language Pre-trained Models for Incremental Face Presentation Attack Detection
Haoze Li, Jie Zhang, Guoying Zhao +2
Face Presentation Attack Detection (PAD) demands incremental learning (IL) to combat evolving spoofing tactics and domains. Privacy regulations, however, forbid retaining past data…
Dual Attention Guided Defense Against Malicious Edits
Jie Zhang, Shuai Dong, Shiguang Shan +1
Recent progress in text-to-image diffusion models has transformed image editing via text prompts, yet this also introduces significant ethical challenges from potential misuse in c…
Semantic Mismatch and Perceptual Degradation: A New Perspective on Image Editing Immunity
Shuai Dong, Jie Zhang, Guoying Zhao +2
Text-guided image editing via diffusion models, while powerful, raises significant concerns about misuse, motivating efforts to immunize images against unauthorized edits using imp…
Towards Transferable Defense Against Malicious Image Edits
Jie Zhang, Shuai Dong, Shiguang Shan +1
Recent approaches employing imperceptible perturbations in input images have demonstrated promising potential to counter malicious manipulations in diffusion-based image editing sy…
Assimilation Matters: Model-level Backdoor Detection in Vision-Language Pretrained Models
Zhongqi Wang, Jie Zhang, Shiguang Shan +1
Vision-language pretrained models (VLPs) such as CLIP have achieved remarkable success, but are also highly vulnerable to backdoor attacks. Given a model fine-tuned by an untrusted…
V-Attack: Targeting Disentangled Value Features for Controllable Adversarial Attacks on LVLMs
Sen Nie, Jie Zhang, Jianxin Yan +2
Adversarial attacks have evolved from simply disrupting predictions on conventional task-specific models to the more complex goal of manipulating image semantics on Large Vision-La…
Dynamic Attention Analysis for Backdoor Detection in Text-to-Image Diffusion Models
Zhongqi Wang, Jie Zhang, Shiguang Shan +1
Recent studies have revealed that text-to-image diffusion models are vulnerable to backdoor attacks, where attackers implant stealthy textual triggers to manipulate model outputs.…
REVAL: A Comprehension Evaluation on Reliability and Values of Large Vision-Language Models
Jie Zhang, Zheng Yuan, Zhongqi Wang +6
The rapid evolution of Large Vision-Language Models (LVLMs) has highlighted the necessity for comprehensive evaluation frameworks that assess these models across diverse dimensions…
Trigger without Trace: Towards Stealthy Backdoor Attack on Text-to-Image Diffusion Models
Jie Zhang, Zhongqi Wang, Shiguang Shan +1
Backdoor attacks targeting text-to-image diffusion models have advanced rapidly. However, current backdoor samples often exhibit two key abnormalities compared to benign samples: 1…
MM-MoralBench: A MultiModal Moral Evaluation Benchmark for Large Vision-Language Models
Bei Yan, Jie Zhang, Zhiyuan Chen +2
The rapid integration of Large Vision-Language Models (LVLMs) into critical domains necessitates comprehensive moral evaluation to ensure their alignment with human values. While e…
Multi-PA: A Multi-perspective Benchmark on Privacy Assessment for Large Vision-Language Models
Jie Zhang, Xiangkui Cao, Zhouyu Han +2
Large Vision-Language Models (LVLMs) exhibit impressive potential across various tasks but also face significant privacy risks, limiting their practical applications. Current resea…
Semantic or Covariate? A Study on the Intractable Case of Out-of-Distribution Detection
Xingming Long, Jie Zhang, Shiguang Shan +1
The primary goal of out-of-distribution (OOD) detection tasks is to identify inputs with semantic shifts, i.e., if samples from novel classes are absent in the in-distribution (ID)…
Confidence Aware Learning for Reliable Face Anti-spoofing
Xingming Long, Jie Zhang, Shiguang Shan
Current Face Anti-spoofing (FAS) models tend to make overly confident predictions even when encountering unfamiliar scenarios or unknown presentation attacks, which leads to seriou…
T2IShield: Defending Against Backdoors on Text-to-Image Diffusion Models
Zhongqi Wang, Jie Zhang, Shiguang Shan +1
While text-to-image diffusion models demonstrate impressive generation capabilities, they also exhibit vulnerability to backdoor attacks, which involve the manipulation of model ou…
Dysca: A Dynamic and Scalable Benchmark for Evaluating Perception Ability of LVLMs
Jie Zhang, Zhongqi Wang, Mengqi Lei +4
Currently many benchmarks have been proposed to evaluate the perception ability of the Large Vision-Language Models (LVLMs). However, most benchmarks conduct questions by selecting…
Measuring the Measurers: Quality Evaluation of Hallucination Benchmarks for Large Vision-Language Models
Bei Yan, Jie Zhang, Zheng Yuan +2
Despite the outstanding performance in multimodal tasks, Large Vision-Language Models (LVLMs) have been plagued by the issue of hallucination, i.e., generating content that is inco…
VLBiasBench: A Comprehensive Benchmark for Evaluating Bias in Large Vision-Language Model
Sibo Wang, Xiangkui Cao, Jie Zhang +4
The emergence of Large Vision-Language Models (LVLMs) marks significant strides towards achieving general artificial intelligence. However, these advancements are accompanied by co…