Causal Evidence for Attention Head Imbalance in Modality Conflict Hallucination

Jinrui Jiang, Zhangtai Wu, Zhen Wu +1

Modality-conflict hallucination occurs when multimodal large language models (MLLMs) prioritize erroneous textual premises over contradictory visual evidence. To understand why vis…