4 papers
Attention Sinks and Outliers in Attention Residuals
Haozheng Luo, Haoran Dai, Shaoyang Zhang +10
We propose OASIS, an outlier- and sink-aware technique built on inter-layer null signaling. As AttnResidual architectures introduce an additional depth-wise normalization channel,…
When One Modality Rules Them All: Backdoor Modality Collapse in Multimodal Diffusion Models
Qitong Wang, Haoran Dai, Haotian Zhang +2
While diffusion models have revolutionized visual content generation, their rapid adoption has underscored the critical need to investigate vulnerabilities, e.g., to backdoor attac…
WildSVG: Towards Reliable SVG Generation Under Real-Word Conditions
Marco Terral, Haotian Zhang, Tianyang Zhang +8
We introduce the task of SVG extraction, which consists in translating specific visual inputs from an image into scalable vector graphics. Existing multimodal models achieve strong…
Practical, Generalizable and Robust Backdoor Attacks on Text-to-Image Diffusion Models
Haoran Dai, Jiawen Wang, Ruo Yang +4
Text-to-image diffusion models (T2I DMs) have achieved remarkable success in generating high-quality and diverse images from text prompts, yet recent studies have revealed their vu…