5 papers
Attention Consistent Longitudinal Medical Visual Question Answering Guided by Vision Foundation Models
Jialin Wu, Qianru Zhang, Georges El Fakhri +1
Longitudinal medical visual question answering (VQA) requires reasoning about anatomical differences between an image of a current time point and an image of a referred time point.…
Revis: Sparse Latent Steering to Mitigate Object Hallucination in Large Vision-Language Models
Jialin Wu, Wei Shi, Han Shen +5
Despite the advanced capabilities of Large Vision-Language Models (LVLMs), they frequently suffer from object hallucination. One reason is that visual features and pretrained textu…
VLMShield: Efficient and Robust Defense of Vision-Language Models against Malicious Prompts
Peigui Qi, Kunsheng Tang, Yanpu Yu +7
Vision-Language Models (VLMs) face significant safety vulnerabilities from malicious prompt attacks due to weakened alignment during visual integration. Existing defenses suffer fr…
Light Alignment Improves LLM Safety via Model Self-Reflection with a Single Neuron
Sicheng Shen, Mingyang Lv, Han Shen +7
The safety of large language models (LLMs) has increasingly emerged as a fundamental aspect of their development. Existing safety alignment for LLMs is predominantly achieved throu…
Saliency Guided Longitudinal Medical Visual Question Answering
Jialin Wu, Xiaofeng Liu
Longitudinal medical visual question answering (Diff-VQA) requires comparing paired studies from different time points and answering questions about clinically meaningful changes.…