13 papers
Thinking Ahead: Foresight Intelligence in MLLMs and World Model
Zhantao Gong, Liaoyuan Fan, Qing Guo +3
In this work, we define Foresight Intelligence as the capability to anticipate and interpret future events-an ability essential for applications such as autonomous driving, yet lar…
VideoScaffold: Elastic-Scale Visual Hierarchies for Streaming Video Understanding in MLLMs
Naishan Zheng, Jie Huang, Qingpei Guo +1
Understanding long videos with multimodal large language models (MLLMs) remains challenging due to the heavy redundancy across frames and the need for temporally coherent represent…
OrdMoE: Preference Alignment via Hierarchical Expert Group Ranking in Multimodal Mixture-of-Experts LLMs
Yuting Gao, Weihao Chen, Lan Wang +2
Preference learning has recently emerged as a pivotal strategy for post-training alignment of Multimodal Large Language Models (MLLMs). However, existing approaches predominantly r…
AnyExperts: On-Demand Expert Allocation for Multimodal Language Models with Mixture of Expert
Yuting Gao, Wang Lan, Hengyuan Zhao +3
Multimodal Mixture-of-Experts (MoE) models offer a promising path toward scalable and efficient large vision-language systems. However, existing approaches rely on rigid routing st…
Ming-UniVision: Joint Image Understanding and Generation with a Unified Continuous Tokenizer
Ziyuan Huang, DanDan Zheng, Cheng Zou +13
Visual tokenization remains a core challenge in unifying visual understanding and generation within the autoregressive paradigm. Existing methods typically employ tokenizers in dis…
VaccineRAG: Boosting Multimodal Large Language Models' Immunity to Harmful RAG Samples
Qixin Sun, Ziqin Wang, Hengyuan Zhao +6
Retrieval Augmented Generation enhances the response accuracy of Large Language Models (LLMs) by integrating retrieval and generation modules with external knowledge, demonstrating…