29 papers
DIVA: Harnessing the Representation Divergence in Unified Multimodal Models for Mutual Reinforcement
Renjie Lu, Xulong Zhang, Xiaoyang Qu +2
Unified Multimodal models (UMMs) built on a single architecture have shown impressive performance in both understanding and generation. We identify a fundamental challenge that lie…
WindowQuant: Mixed-Precision KV Cache Quantization based on Window-Level Similarity for VLMs Inference Optimization
Wei Tao, Xiaoyang Qu, Peiqiang Wang +4
Recently, video language models (VLMs) have been applied in various fields. However, the visual token sequence of the VLM is too long, which may cause intolerant inference latency…
From Inheritance to Saturation: Disentangling the Evolution of Visual Redundancy for Architecture-Aware MLLM Inference Acceleration
Jiaqi Shi, Yuechan Li, Xulong Zhang +2
High-resolution Multimodal Large Language Models (MLLMs) face prohibitive computational costs during inference due to the explosion of visual tokens. Existing acceleration strategi…
VLA-InfoEntropy: A Training-Free Vision-Attention Information Entropy Approach for Vision-Language-Action Models Inference Acceleration and Success
Chuhang Liu, Yayun He, Zuheng Kang +2
Vision-Language-Action (VLA) models integrate visual perception, language understanding, and action decision-making for cross-modal semantic alignment, exhibiting broad application…
Vista: Scene-Aware Optimization for Streaming Video Question Answering under Post-Hoc Queries
Haocheng Lu, Nan Zhang, Wei Tao +4
Streaming video question answering (Streaming Video QA) poses distinct challenges for multimodal large language models (MLLMs), as video frames arrive sequentially and user queries…
From Knowing to Doing Precisely: A General Self-Correction and Termination Framework for VLA models
Wentao Zhang, Aolan Sun, Wentao Mo +3
While vision-language-action (VLA) models for embodied agents integrate perception, reasoning, and control, they remain constrained by two critical weaknesses: first, during graspi…