7 papers
SafeCA: Safe Cross-Attention Localization and Regulation for Text-to-Video Jailbreak Defense
Siyuan Liang, Yupeng Qiu, Junfeng Fang +3
Text-to-Video (T2V) generative models are vulnerable to jailbreak attacks in real-world deployment, leading them to produce harmful or inappropriate content. Existing defense appro…
Mitigating Hallucinations in Large Vision-Language Models without Performance Degradation
Xingyu Zhu, Junfeng Fang, Shuo Wang +4
Large Vision-Language Models (LVLMs) exhibit powerful generative capabilities but frequently produce hallucinations that compromise output reliability. Fine-tuning on annotated dat…
Principled Steering via Null-space Projection for Jailbreak Defense in Vision-Language Models
Xingyu Zhu, Beier Zhu, Shuo Wang +4
As vision-language models (VLMs) are increasingly deployed in open-world scenarios, they can be easily induced by visual jailbreak attacks to generate harmful content, posing serio…
GuardAlign: Test-time Safety Alignment in Multimodal Large Language Models
Xingyu Zhu, Beier Zhu, Junfeng Fang +4
Large vision-language models (LVLMs) have achieved remarkable progress in vision-language reasoning tasks, yet ensuring their safety remains a critical challenge. Recent input-side…
Revisiting Robustness for LLM Safety Alignment via Selective Geometry Control
Yonghui Yang, Wenjian Tao, Jilong Liu +6
Safety alignment of large language models remains brittle under domain shift and noisy preference supervision. Most existing robust alignment methods focus on uncertainty in alignm…
Hierarchical Semantic Alignment for Image Clustering
Xingyu Zhu, Beier Zhu, Yunfan Li +4
Image clustering is a classic problem in computer vision, which categorizes images into different groups. Recent studies utilize nouns as external semantic knowledge to improve clu…
Dynamic Multimodal Prototype Learning in Vision-Language Models
Xingyu Zhu, Shuo Wang, Beier Zhu +6
With the increasing attention to pre-trained vision-language models (VLMs), \eg, CLIP, substantial efforts have been devoted to many downstream tasks, especially in test-time adapt…