6 papers · 1 filter
Uncovering Understanding-Generation Synergy in Native Unified Multimodal Models: From Representation, Task to System
Penghao Wu, Haiwen Diao, Weichen Fan +3
While unified multimodal models (UMMs) jointly perform visual understanding and generation within a single model, functional unification does not guarantee learning synergy: the tw…
Scaling Spatial Intelligence with Multimodal Foundation Models
Zhongang Cai, Ruisi Wang, Chenyang Gu +26
Despite remarkable progress, multimodal foundation models still exhibit surprising deficiencies in spatial intelligence. In this work, we explore scaling up multimodal foundation m…
From Pixels to Words -- Towards Native Vision-Language Primitives at Scale
Haiwen Diao, Mingxuan Li, Silei Wu +6
The edifice of native Vision-Language Models (VLMs) has emerged as a rising contender to typical modular VLMs, shaped by evolving model architectures and training paradigms. Yet, t…
Visual Jigsaw Post-Training Improves MLLMs
Penghao Wu, Yushan Zhang, Haiwen Diao +3
Reinforcement learning based post-training has recently emerged as a powerful paradigm for enhancing the alignment and reasoning capabilities of multimodal large language models (M…
Holistic Evaluation of Multimodal LLMs on Spatial Intelligence
Zhongang Cai, Yubo Wang, Qingping Sun +21
Multimodal models have achieved remarkable progress in recent years. Nevertheless, they continue to exhibit notable limitations in spatial understanding and reasoning, the very cap…
Streamline Without Sacrifice -- Squeeze out Computation Redundancy in LMM
Penghao Wu, Lewei Lu, Ziwei Liu
Large multimodal models excel in multimodal tasks but face significant computational challenges due to excessive computation on visual tokens. Unlike token reduction methods that f…