15 papers
GE-Act 2.0: Pretraining and Scaling a World-Action Model for Robotic Manipulation
AgiBot Research Team, Renhang Liu, Wenzhi Zhao +42
World-action models (WAM) predict future states to guide robot actions, enabling learning from both action-free video and action-labeled interaction. Most inherit pretrained video…
Video-IFBench: Evaluating Instruction Following of Multimodal LLMs in Video Understanding Scenarios
Hongbo Liu, Peixian Chen, Sihan Liu +12
Multimodal Large Language Models (MLLMs) have shown strong performance in video understanding. However, their ability to follow instructions in this domain remains under-explored.…
FIRM-Video: Check Before You Score for Reliable Text-to-Video Reward Modeling
Peiyuan Zhang, Xiangyu Zhao, Hongbo Liu +8
Reliable reward models are essential for text-to-video evaluation and alignment. However, the trade-off between evaluation accuracy and inference efficiency places high demands on…
Moment-Video: Diagnosing Temporal Fidelity of Video MLLMs on Momentary Visual Events
Xiaolin Liu, Yilun Zhu, Xiangyu Zhao +9
Video multimodal large language models (MLLMs) have made rapid progress in general and long-form video understanding, yet their ability to capture and use brief, answer-critical vi…
Video-MME-v2: Towards the Next Stage in Benchmarks for Comprehensive Video Understanding
Chaoyou Fu, Haozhi Yuan, Yuhao Dong +16
With the rapid advancement of video understanding, existing benchmarks are becoming increasingly saturated, exposing a critical discrepancy between inflated leaderboard scores and…
Dynamic Token Compression for Efficient Video Understanding through Reinforcement Learning
Shida Wang, YongXiang Hua, Zhou Tao +2
Multimodal Large Language Models have demonstrated remarkable capabilities in video understanding, yet face prohibitive computational costs and performance degradation from ''conte…
Omni-Diffusion: Unified Multimodal Understanding and Generation with Masked Discrete Diffusion
Lijiang Li, Zuwei Long, Yunhang Shen +6
While recent multimodal large language models (MLLMs) have made impressive strides, they predominantly employ a conventional autoregressive architecture as their backbone, leaving…
Can Unified Generation and Understanding Models Maintain Semantic Equivalence Across Different Output Modalities?
Hongbo Jiang, Jie Li, Yunhang Shen +4
Unified Multimodal Large Language Models (U-MLLMs) integrate understanding and generation within a single architecture. However, existing evaluations typically assess these capabil…
RISE-Video: Can Video Generators Decode Implicit World Rules?
Mingxin Liu, Shuran Ma, Shibei Meng +9
While generative video models have achieved remarkable visual fidelity, their capacity to internalize and reason over implicit world rules remains a critical yet under-explored fro…
Youtu-VL: Unleashing Visual Potential via Unified Vision-Language Supervision
Zhixiang Wei, Yi Li, Zhehan Kan +38
Despite the significant advancements represented by Vision-Language Models (VLMs), current architectures often exhibit limitations in retaining fine-grained visual information, lea…
Youtu-Parsing: Perception, Structuring and Recognition via High-Parallelism Decoding
Haoyu Cao, Kun Yin, Yunfei Wu +16
This paper presents Youtu-Parsing, an efficient and versatile document parsing model designed for high-performance content extraction. The architecture employs a native Vision Tran…
TangramPuzzle: Evaluating Multimodal Large Language Models with Compositional Spatial Reasoning
Daixian Liu, Jiayi Kuang, Yinghui Li +8
Multimodal Large Language Models (MLLMs) have achieved remarkable progress in visual recognition and semantic understanding, yet precise compositional spatial reasoning under geome…
VITA-VLA: Efficiently Teaching Vision-Language Models to Act via Action Expert Distillation
Shaoqi Dong, Chaoyou Fu, Haihan Gao +12
Vision-Language Action (VLA) models significantly advance robotic manipulation by leveraging the strong perception capabilities of pretrained vision-language models (VLMs). By inte…
VITA-Audio: Fast Interleaved Cross-Modal Token Generation for Efficient Large Speech-Language Model
Zuwei Long, Yunhang Shen, Chaoyou Fu +11
With the growing requirement for natural human-computer interaction, speech-based systems receive increasing attention as speech is one of the most common forms of daily communicat…
Long-VITA: Scaling Large Multi-modal Models to 1 Million Tokens with Leading Short-Context Accuracy
Yunhang Shen, Chaoyou Fu, Shaoqi Dong +14
We introduce Long-VITA, a simple yet effective large multi-modal model for long-context visual-language understanding tasks. It is adept at concurrently processing and analyzing mo…