5 papers
Under the Skin: Backdoor Infection in Multimodal Large Language Models
Jiawen Wang, Ruo Yang, Haoran Dai +2
Multimodal Large Language Models (MLLMs) have achieved impressive performance in generating text conditioned on inputs from different modalities. In this work, we introduce backdoo…
Attention Sinks and Outliers in Attention Residuals
Haozheng Luo, Haoran Dai, Shaoyang Zhang +10
We propose OASIS, an outlier- and sink-aware method that stabilizes dual-normalized attention-residual architectures through explicit null routing and token-to-depth null coupling.…
When One Modality Rules Them All: Backdoor Modality Collapse in Multimodal Diffusion Models
Qitong Wang, Haoran Dai, Haotian Zhang +2
While diffusion models have revolutionized visual content generation, their rapid adoption has underscored the critical need to investigate vulnerabilities, e.g., to backdoor attac…
WildSVG: Towards Reliable SVG Generation Under Real-Word Conditions
Marco Terral, Haotian Zhang, Tianyang Zhang +8
We introduce the task of SVG extraction, which consists in translating specific visual inputs from an image into scalable vector graphics. Existing multimodal models achieve strong…
Practical, Generalizable and Robust Backdoor Attacks on Text-to-Image Diffusion Models
Haoran Dai, Jiawen Wang, Ruo Yang +4
Text-to-image diffusion models (T2I DMs) have achieved remarkable success in generating high-quality and diverse images from text prompts, yet recent studies have revealed their vu…