Strong Helps Weak: Directional Cross-Modal Alignment Transfer in Multi-modal LLMs

Hoigi Seo, Byung Hyun Lee, Minjun Kim +3

Multi-modal large language models (MLLMs) achieve strong modality understanding by pairing a large language model (LLM) with an encoder for a target modality such as vision, video,…