6 papers
From Pixels to Words -- Towards Native One-Vision Models at Scale
Haiwen Diao, Jiahao Wang, Penghao Wu +18
Current vision-language models (VLMs) typically stitch together separate image encoders and language decoders via multi-stage alignment, a modular framework that inevitably fragmen…
Visual Jigsaw Post-Training Improves MLLMs
Penghao Wu, Yushan Zhang, Haiwen Diao +3
Reinforcement learning based post-training has recently emerged as a powerful paradigm for enhancing the alignment and reasoning capabilities of multimodal large language models (M…
Ego-R1: Chain-of-Tool-Thought for Ultra-Long Egocentric Video Reasoning
Shulin Tian, Ruiqi Wang, Hongming Guo +7
We introduce Ego-R1, a novel framework for reasoning over ultra-long (i.e., in days and weeks) egocentric videos, which leverages a structured Chain-of-Tool-Thought (CoTT) process,…
GUI-Reflection: Empowering Multimodal GUI Models with Self-Reflection Behavior
Penghao Wu, Shengnan Ma, Bo Wang +3
Multimodal Large Language Models (MLLMs) have shown great potential in revolutionizing Graphical User Interface (GUI) automation. However, existing GUI models mostly rely on learni…
Streamline Without Sacrifice -- Squeeze out Computation Redundancy in LMM
Penghao Wu, Lewei Lu, Ziwei Liu
Large multimodal models excel in multimodal tasks but face significant computational challenges due to excessive computation on visual tokens. Unlike token reduction methods that f…
Video-MMMU: Evaluating Knowledge Acquisition from Multi-Discipline Professional Videos
Kairui Hu, Penghao Wu, Fanyi Pu +5
Humans acquire knowledge through three cognitive stages: perceiving information, comprehending knowledge, and adapting knowledge to solve novel problems. Videos serve as an effecti…