4 citations · 22 across the 29 of their papers we have counts for
17 papers · 1 filter
Seeing Through the Chain: Understanding and Mitigating Hallucinations in Multimodal Large Reasoning Models
Hao Fang, Jinyu Li, Jiawei Kong +4
While multimodal reasoning models (MLRMs) have exhibited impressive capabilities, they remain prone to hallucinations, and effective solutions are still underexplored. In this pape…
Pre-training CLIP against Data Poisoning with Optimal Transport-based Matching and Alignment
Tong Zhang, Kuofeng Gao, Jiawang Bai +5
Recent studies have shown that Contrastive Language-Image Pre-training (CLIP) models are threatened by targeted data poisoning and backdoor attacks due to massive training image-ca…
Towards Dataset Copyright Evasion Attack against Personalized Text-to-Image Diffusion Models
Kuofeng Gao, Yufei Zhu, Yiming Li +4
Text-to-image (T2I) diffusion models enable high-quality image generation conditioned on textual prompts. However, fine-tuning these pre-trained models for personalization raises c…
Protecting Your Video Content: Disrupting Automated Video-based LLM Annotations
Haitong Liu, Kuofeng Gao, Yang Bai +4
Recently, video-based large language models (video-based LLMs) have achieved impressive performance across various video comprehension tasks. However, this rapid advancement raises…
Neural Antidote: Class-Wise Prompt Tuning for Purifying Backdoors in CLIP
Jiawei Kong, Hao Fang, Sihang Guo +5
While pre-trained Vision-Language Models (VLMs) such as CLIP exhibit impressive representational capabilities for multimodal data, recent studies have revealed their vulnerability…
Retrievals Can Be Detrimental: Unveiling the Backdoor Vulnerability of Retrieval-Augmented Diffusion Models
Hao Fang, Xiaohang Sui, Hongyao Yu +5
Diffusion models (DMs) have recently demonstrated remarkable generation capability. However, their training generally requires huge computational resources and large-scale datasets…
Video Watermarking: Safeguarding Your Video from (Unauthorized) Annotations by Video-based LLMs
Jinmin Li, Kuofeng Gao, Yang Bai +2
The advent of video-based Large Language Models (LLMs) has significantly enhanced video understanding. However, it has also raised some safety concerns regarding data protection, a…
Not All Prompts Are Secure: A Switchable Backdoor Attack Against Pre-trained Vision Transformers
Sheng Yang, Jiawang Bai, Kuofeng Gao +3
Given the power of vision transformers, a new learning paradigm, pre-training and then prompting, makes it more efficient and effective to address downstream visual recognition tas…
Adversarial Robustness for Visual Grounding of Multimodal Large Language Models
Kuofeng Gao, Yang Bai, Jiawang Bai +2
Multi-modal Large Language Models (MLLMs) have recently achieved enhanced performance across various vision-language tasks including visual grounding capabilities. However, the adv…
Energy-Latency Manipulation of Multi-modal Large Language Models via Verbose Samples
Kuofeng Gao, Jindong Gu, Yang Bai +4
Despite the exceptional performance of multi-modal large language models (MLLMs), their deployment requires substantial computational resources. Once malicious users induce high en…
FMM-Attack: A Flow-based Multi-modal Adversarial Attack on Video-based LLMs
Jinmin Li, Kuofeng Gao, Yang Bai +3
Despite the remarkable performance of video-based large language models (LLMs), their adversarial threat remains unexplored. To fill this gap, we propose the first adversarial atta…
Inducing High Energy-Latency of Large Vision-Language Models with Verbose Images
Kuofeng Gao, Yang Bai, Jindong Gu +4
Large vision-language models (VLMs) such as GPT-4 have achieved exceptional performance across various multi-modal tasks. However, the deployment of VLMs necessitates substantial e…
BadCLIP: Trigger-Aware Prompt Learning for Backdoor Attacks on CLIP
Jiawang Bai, Kuofeng Gao, Shaobo Min +3
Contrastive Vision-Language Pre-training, known as CLIP, has shown promising effectiveness in addressing downstream image recognition tasks. However, recent works revealed that the…
Backdoor Defense via Adaptively Splitting Poisoned Dataset
Kuofeng Gao, Yang Bai, Jindong Gu +2
Backdoor defenses have been studied to alleviate the threat of deep neural networks (DNNs) being backdoor attacked and thus maliciously altered. Since DNNs usually adopt some exter…
Imperceptible and Robust Backdoor Attack in 3D Point Cloud
Kuofeng Gao, Jiawang Bai, Baoyuan Wu +2
With the thriving of deep learning in processing point cloud data, recent works show that backdoor attacks pose a severe security threat to 3D vision applications. The attacker inj…
Hardly Perceptible Trojan Attack against Neural Networks with Bit Flips
Jiawang Bai, Kuofeng Gao, Dihong Gong +3
The security of deep neural networks (DNNs) has attracted increasing attention due to their widespread use in various applications. Recently, the deployed DNNs have been demonstrat…
Backdoor Attack on Hash-based Image Retrieval via Clean-label Data Poisoning
Kuofeng Gao, Jiawang Bai, Bin Chen +2
A backdoored deep hashing model is expected to behave normally on original query images and return the images with the target label when a specific trigger pattern presents. To thi…