7 citations · 12 across the 26 of their papers we have counts for
31 papers
MergeHEIR: Mitigating Multimodal Hallucinations as the Tax of Model Merging
Jinyu Li, Hao Fang, Zhiming Zhang +3
Model merging consolidates task-specialized experts into a single deployable model. However, we show that such capability consolidation incurs a merging tax of increased hallucinat…
From Bits to Beliefs: Recoverable Semantic Fingerprints for Black-Box Verification of Large Language Models
Jiaxin Hong, Yuxin Peng, Hongyao Yu +3
Open-weight large language models (LLMs) can be copied, modified, and redeployed behind black-box APIs, making post-release ownership verification difficult. Existing black-box fin…
Weak Ties, Strong Signals: Efficient Training Data Detection in Diffusion LLMs via Independent Token Sampling
Hongyao Yu, Tianqu Zhuang, Ziyuan Xu +4
Diffusion large language models (dLLMs) offer a compelling alternative to autoregressive models, yet they may expose sensitive training data during denoising. Detecting such usage…
LEMUR: Latent Entropy-aware Multimodal Unlearning via Visual-anchored Reasoning Redirection
Xinhao Zhong, Yuxia Qiao, Junhao Li +3
Reinforcement-learning (RL) post-training equips multimodal large reasoning models (MLRMs) with exploratory chains of thought (CoT), substantially improving visual reasoning. Howev…
CHILLGuard: Towards Fine-Grained Chinese LLM Safety Guardrail with Scalable Data Construction and Model-aware Preference Alignment
Wenbo Yu, Bohua Wang, Hao Fang +10
Malicious content generated from large language models (LLMs) could pose severe safety risks and ethical concerns. While existing LLM safety guardrails excel in English or multilin…
MemVenom: Triggered Poisoning of Multimodal Memories in Web Agents
Yv Zhang, Hao Sun, Hao Fang +5
External memory has become a core component of modern web agents, enabling long-horizon reasoning through the retrieval of past experiences. However, this paradigm introduces a cri…
Bypassing Copyright Protection in Diffusion-based Customization via Two-Stage Latent Feature Optimization
Ziang Xu, Wenbo Yu, Hongyao Yu +6
With the growing concerns over copyright infringement in diffusion-based customization, adversarial attacks have emerged as a prominent defense strategy to prevent malicious conten…
Reasoning Matters: Mitigate Hallucination in Multimodal Large Reasoning Models via Reasoning-Conditioned Preference Optimization
Jiawei Kong, Hao Fang, Shunxiang Liao +5
Multimodal Large Reasoning Models introduce the reasoning paradigm, demonstrating strong capabilities on complex vision-language tasks. However, they still suffer from severe hallu…
Prompt2Fingerprint: Plug-and-Play LLM Fingerprinting via Text-to-Weight Generation
Sixu Chen, Xiang Chen, Hongyao Yu +5
The widespread deployment and redistribution of large language models (LLMs) have made model provenance tracking a critical challenge. While existing LLM fingerprinting methods, pa…
Mistletoe: Stealthy Acceleration-Collapse Attacks on Speculative Decoding
Shuoyang Sun, Chang Dai, Hao Fang +6
Speculative decoding has become a widely adopted technique for accelerating large language model (LLM) inference by drafting multiple candidate tokens and verifying them with a tar…
Seeing Through the Chain: Understanding and Mitigating Hallucinations in Multimodal Large Reasoning Models
Hao Fang, Jinyu Li, Jiawei Kong +4
While multimodal reasoning models (MLRMs) have exhibited impressive capabilities, they remain prone to hallucinations, and effective solutions are still underexplored. In this pape…
Enhancing Gradient Inversion Attacks in Federated Learning via Hierarchical Feature Optimization
Hao Fang, Wenbo Yu, Bin Chen +4
Federated Learning (FL) has emerged as a compelling paradigm for privacy-preserving distributed machine learning, allowing multiple clients to collaboratively train a global model…
Looking Back and Forth: Cross-Image Attention Calibration and Attentive Preference Learning for Multi-Image Hallucination Mitigation
Xiaochen Yang, Hao Fang, Jiawei Kong +3
Although large vision-language models (LVLMs) have demonstrated remarkable capabilities, they are prone to hallucinations in multi-image tasks. We attribute this issue to limitatio…
Towards Distillation-Resistant Large Language Models: An Information-Theoretic Perspective
Hao Fang, Tianyi Zhang, Tianqu Zhuang +6
Proprietary large language models (LLMs) embody substantial economic value and are generally exposed only as black-box APIs, yet adversaries can still exploit their outputs to extr…
Revisiting the Privacy Risks of Split Inference: A GAN-Based Data Reconstruction Attack via Progressive Feature Optimization
Yixiang Qiu, Yanhan Liu, Hongyao Yu +4
The growing complexity of Deep Neural Networks (DNNs) has led to the adoption of Split Inference (SI), a collaborative paradigm that partitions computation between edge devices and…
ICAS: Detecting Training Data from Autoregressive Image Generative Models
Hongyao Yu, Yixiang Qiu, Yiheng Yang +6
Autoregressive image generation has witnessed rapid advancements, with prominent models such as scale-wise visual auto-regression pushing the boundaries of visual synthesis. Howeve…
Grounding Language with Vision: A Conditional Mutual Information Calibrated Decoding Strategy for Reducing Hallucinations in LVLMs
Hao Fang, Changle Zhou, Jiawei Kong +3
Large Vision-Language Models (LVLMs) are susceptible to hallucinations, where generated responses seem semantically plausible yet exhibit little or no relevance to the input image.…
Revisiting Backdoor Attacks on LLMs: A Stealthy and Practical Poisoning Framework via Harmless Inputs
Jiawei Kong, Hao Fang, Xiaochen Yang +5
Recent studies have widely investigated backdoor attacks on Large Language Models (LLMs) by inserting harmful question-answer (QA) pairs into their training data. However, we revis…
Your Language Model Can Secretly Write Like Humans: Contrastive Paraphrase Attacks on LLM-Generated Text Detectors
Hao Fang, Jiawei Kong, Tianqu Zhuang +6
The misuse of large language models (LLMs), such as academic plagiarism, has driven the development of detectors to identify LLM-generated texts. To bypass these detectors, paraphr…
GaussTrap: Stealthy Backdoor Attacks on 3D Gaussian Splatting for Targeted Scene Misperception
Jiaxin Hong, Sixu Chen, Shuoyang Sun +7
3D Gaussian Splatting (3DGS) has recently emerged as a powerful paradigm for real-time scene representation and novel view synthesis, gaining traction in safety-critical applicatio…
Neural Antidote: Class-Wise Prompt Tuning for Purifying Backdoors in CLIP
Jiawei Kong, Hao Fang, Sihang Guo +5
While pre-trained Vision-Language Models (VLMs) such as CLIP exhibit impressive representational capabilities for multimodal data, recent studies have revealed their vulnerability…
Retrievals Can Be Detrimental: Unveiling the Backdoor Vulnerability of Retrieval-Augmented Diffusion Models
Hao Fang, Xiaohang Sui, Hongyao Yu +5
Diffusion models (DMs) have recently demonstrated remarkable generation capability. However, their training generally requires huge computational resources and large-scale datasets…
Going Beyond Feature Similarity: Effective Dataset Distillation based on Class-Aware Conditional Mutual Information
Xinhao Zhong, Bin Chen, Hao Fang +3
Dataset distillation (DD) aims to minimize the time and memory consumption needed for training deep neural networks on large datasets, by creating a smaller synthetic dataset that…
Rank Matters: Understanding and Defending Model Inversion Attacks via Low-Rank Feature Filtering
Hongyao Yu, Yixiang Qiu, Hao Fang +6
Model Inversion Attacks (MIAs) pose a significant threat to data privacy by reconstructing sensitive training samples from the knowledge embedded in trained machine learning models…