Not All Modalities Are Equal: Instruction-Aware Gating for Multimodal Videos

Bonan Ding, Umair Nawaz, Ufaq Khan +5

Pre-trained video large language models excel at visual reasoning. However, they struggle when videos arrive with auxiliary streams, such as audio, depth map, or dense temporal evi…