ReUnit: Multi-Granularity Visual Unitization for Long Video Understanding

Abstract

Long-video understanding is constrained by the limited visual input capacity of video multimodal large language models (Video-MLLMs). Existing methods mainly optimize which content to retain, while the presentation of retained content often remains fixed. As a result, the same balance between spatial detail and content coverage is imposed across the entire visual input. We propose ReUnit, a training-free and query-aware framework that jointly determines which content to retain and how it should be presented. Guided by frame-level query relevance, ReUnit constructs and allocates visual units at multiple presentation granularities. More relevant content receives finer presentation, while broader context is represented more compactly. It realizes these granularities using visual units that carry one, four, or nine source frames and are each rendered as a standard image. Across four benchmarks and visual input budgets from 4 to 32 units, ReUnit achieves the highest average score at every tested budget. It improves over Uniform Sampling by 8.3--10.1 points on average, and the gains transfer across three additional Video-MLLM families. Project resources are available at https://github.com/charon525/ReUnit.

Major revision of the original Q-Fold manuscript. Renamed to ReUnit with reformulated methodology, expanded experiments, and additional analysis