Mitigating Cross-Modal Hallucinations by Correcting Modality Misrouting in Omnimodal LLMs

Yu Mi, Licheng Pan, Jiayi Zhu +7

Omnimodal large language models (OLLMs) are susceptible to cross-modal hallucinations under audio-visual interference. We find that, for a substantial fraction of incorrect joint p…