6 papers
Scaling Spatial Intelligence with Multimodal Foundation Models
Zhongang Cai, Ruisi Wang, Chenyang Gu +26
Despite remarkable progress, multimodal foundation models still exhibit surprising deficiencies in spatial intelligence. In this work, we explore scaling up multimodal foundation m…
From Pixels to Words -- Towards Native Vision-Language Primitives at Scale
Haiwen Diao, Mingxuan Li, Silei Wu +6
The edifice of native Vision-Language Models (VLMs) has emerged as a rising contender to typical modular VLMs, shaped by evolving model architectures and training paradigms. Yet, t…
Holistic Evaluation of Multimodal LLMs on Spatial Intelligence
Zhongang Cai, Yubo Wang, Qingping Sun +21
Multimodal models have achieved remarkable progress in recent years. Nevertheless, they continue to exhibit notable limitations in spatial understanding and reasoning, the very cap…
Visual Jigsaw Post-Training Improves MLLMs
Penghao Wu, Yushan Zhang, Haiwen Diao +3
Reinforcement learning based post-training has recently emerged as a powerful paradigm for enhancing the alignment and reasoning capabilities of multimodal large language models (M…
GUI-Reflection: Empowering Multimodal GUI Models with Self-Reflection Behavior
Penghao Wu, Shengnan Ma, Bo Wang +3
Multimodal Large Language Models (MLLMs) have shown great potential in revolutionizing Graphical User Interface (GUI) automation. However, existing GUI models mostly rely on learni…
Streamline Without Sacrifice -- Squeeze out Computation Redundancy in LMM
Penghao Wu, Lewei Lu, Ziwei Liu
Large multimodal models excel in multimodal tasks but face significant computational challenges due to excessive computation on visual tokens. Unlike token reduction methods that f…