4 citations · 22 across the 29 of their papers we have counts for
33 papers
Information Blackhole: Exploring Backdoor Mechanism in 3D Point Cloud Reconstruction
Zhifei Yang, Xiuping Liu, Kuofeng Gao +3
Point cloud autoencoders are fundamental components for 3D world representation and support many safety-critical downstream applications. Existing studies have extensively investig…
Balancing Efficiency and Efficacy: Training-Free Attention-Guided Switching Between Explicit and Latent Thoughts for MLLMs
Haoqian Kang, Liupeng Li, Kuofeng Gao +5
Reasoning in Multimodal Large Language Models (MLLMs) requires both fine-grained visual perception and rigorous logical deduction. Explicit text-based Chain-of-Thought (CoT) is com…
CHILLGuard: Towards Fine-Grained Chinese LLM Safety Guardrail with Scalable Data Construction and Model-aware Preference Alignment
Wenbo Yu, Bohua Wang, Hao Fang +10
Malicious content generated from large language models (LLMs) could pose severe safety risks and ethical concerns. While existing LLM safety guardrails excel in English or multilin…
MemVenom: Triggered Poisoning of Multimodal Memories in Web Agents
Yv Zhang, Hao Sun, Hao Fang +5
External memory has become a core component of modern web agents, enabling long-horizon reasoning through the retrieval of past experiences. However, this paradigm introduces a cri…
Mistletoe: Stealthy Acceleration-Collapse Attacks on Speculative Decoding
Shuoyang Sun, Chang Dai, Hao Fang +6
Speculative decoding has become a widely adopted technique for accelerating large language model (LLM) inference by drafting multiple candidate tokens and verifying them with a tar…
Seeing Through the Chain: Understanding and Mitigating Hallucinations in Multimodal Large Reasoning Models
Hao Fang, Jinyu Li, Jiawei Kong +4
While multimodal reasoning models (MLRMs) have exhibited impressive capabilities, they remain prone to hallucinations, and effective solutions are still underexplored. In this pape…
Towards Distillation-Resistant Large Language Models: An Information-Theoretic Perspective
Hao Fang, Tianyi Zhang, Tianqu Zhuang +6
Proprietary large language models (LLMs) embody substantial economic value and are generally exposed only as black-box APIs, yet adversaries can still exploit their outputs to extr…
JPRO: Automated Multimodal Jailbreaking via Multi-Agent Collaboration Framework
Yuxuan Zhou, Yang Bai, Kuofeng Gao +2
The widespread application of large VLMs makes ensuring their secure deployment critical. While recent studies have demonstrated jailbreak attacks on VLMs, existing approaches are…
Why Does Weak-OOD Help? A Further Step Towards Understanding Jailbreaking VLMs
Yuxuan Zhou, Yuzhao Peng, Yang Bai +7
Large Vision-Language Models (VLMs) are susceptible to jailbreak attacks: researchers have developed various attack strategies that bypass the safety mechanisms of VLMs. Among thes…
Imperceptible Jailbreaking against Large Language Models
Kuofeng Gao, Yiming Li, Chao Du +4
Jailbreaking attacks on the vision modality typically rely on imperceptible adversarial perturbations, whereas attacks on the textual modality are generally assumed to require visi…
Pre-training CLIP against Data Poisoning with Optimal Transport-based Matching and Alignment
Tong Zhang, Kuofeng Gao, Jiawang Bai +5
Recent studies have shown that Contrastive Language-Image Pre-training (CLIP) models are threatened by targeted data poisoning and backdoor attacks due to massive training image-ca…
Grounding Language with Vision: A Conditional Mutual Information Calibrated Decoding Strategy for Reducing Hallucinations in LVLMs
Hao Fang, Changle Zhou, Jiawei Kong +3
Large Vision-Language Models (LVLMs) are susceptible to hallucinations, where generated responses seem semantically plausible yet exhibit little or no relevance to the input image.…
Revisiting Backdoor Attacks on LLMs: A Stealthy and Practical Poisoning Framework via Harmless Inputs
Jiawei Kong, Hao Fang, Xiaochen Yang +5
Recent studies have widely investigated backdoor attacks on Large Language Models (LLMs) by inserting harmful question-answer (QA) pairs into their training data. However, we revis…
Your Language Model Can Secretly Write Like Humans: Contrastive Paraphrase Attacks on LLM-Generated Text Detectors
Hao Fang, Jiawei Kong, Tianqu Zhuang +6
The misuse of large language models (LLMs), such as academic plagiarism, has driven the development of detectors to identify LLM-generated texts. To bypass these detectors, paraphr…
Towards Dataset Copyright Evasion Attack against Personalized Text-to-Image Diffusion Models
Kuofeng Gao, Yufei Zhu, Yiming Li +4
Text-to-image (T2I) diffusion models enable high-quality image generation conditioned on textual prompts. However, fine-tuning these pre-trained models for personalization raises c…
Protecting Your Video Content: Disrupting Automated Video-based LLM Annotations
Haitong Liu, Kuofeng Gao, Yang Bai +4
Recently, video-based large language models (video-based LLMs) have achieved impressive performance across various video comprehension tasks. However, this rapid advancement raises…
Neural Antidote: Class-Wise Prompt Tuning for Purifying Backdoors in CLIP
Jiawei Kong, Hao Fang, Sihang Guo +5
While pre-trained Vision-Language Models (VLMs) such as CLIP exhibit impressive representational capabilities for multimodal data, recent studies have revealed their vulnerability…
Retrievals Can Be Detrimental: Unveiling the Backdoor Vulnerability of Retrieval-Augmented Diffusion Models
Hao Fang, Xiaohang Sui, Hongyao Yu +5
Diffusion models (DMs) have recently demonstrated remarkable generation capability. However, their training generally requires huge computational resources and large-scale datasets…
Benchmarking Open-ended Audio Dialogue Understanding for Large Audio-Language Models
Kuofeng Gao, Shu-Tao Xia, Ke Xu +2
Large Audio-Language Models (LALMs), such as GPT-4o, have recently unlocked audio dialogue capabilities, enabling direct spoken exchanges with humans. The potential of LALMs broade…
Denial-of-Service Poisoning Attacks against Large Language Models
Kuofeng Gao, Tianyu Pang, Chao Du +3
Recent studies have shown that LLMs are vulnerable to denial-of-service (DoS) attacks, where adversarial inputs like spelling errors or non-semantic prompts trigger endless outputs…
Embedding Self-Correction as an Inherent Ability in Large Language Models for Enhanced Mathematical Reasoning
Kuofeng Gao, Huanqia Cai, Qingyao Shuai +2
Accurate mathematical reasoning with Large Language Models (LLMs) is crucial in revolutionizing domains that heavily rely on such reasoning. However, LLMs often encounter difficult…
PointNCBW: Towards Dataset Ownership Verification for Point Clouds via Negative Clean-label Backdoor Watermark
Cheng Wei, Yang Wang, Kuofeng Gao +4
Recently, point clouds have been widely used in computer vision, whereas their collection is time-consuming and expensive. As such, point cloud datasets are the valuable intellectu…
Video Watermarking: Safeguarding Your Video from (Unauthorized) Annotations by Video-based LLMs
Jinmin Li, Kuofeng Gao, Yang Bai +2
The advent of video-based Large Language Models (LLMs) has significantly enhanced video understanding. However, it has also raised some safety concerns regarding data protection, a…
Not All Prompts Are Secure: A Switchable Backdoor Attack Against Pre-trained Vision Transformers
Sheng Yang, Jiawang Bai, Kuofeng Gao +3
Given the power of vision transformers, a new learning paradigm, pre-training and then prompting, makes it more efficient and effective to address downstream visual recognition tas…