12 papers · 1 filter
OmniAssistBench: Assistant-style Interaction Benchmark for Omni-LLMs
Xianyun Sun, Chaoyou Fu, Zhengye Zhang +6
Recent omni-modal large language models (Omni-LLMs) show great potential as real-time video assistants, which continuously perceive environments and guide users to achieve specific…
Omni-Prune: Query-Aware Unified Token Pruning for Efficient Omnimodal Large Language Models
Yiming Zhong, Chang Nie, Caifeng Shan
Omnimodal large language models (OmniLLMs) are rapidly extending multimodal reasoning to cover synchronized audio and video. However, the resulting audio-video token sequences are…
Light-Omni: Reflex over Reasoning in Agentic Video Understanding with Long-Term Memory
Chang Nie, Jiaju Wei, Junlan Feng +2
Agentic video understanding equips models with long-term memory to autonomously process and respond to continuous, long-horizon multimodal streams. However, advanced video agents o…
Omni-Diffusion: Unified Multimodal Understanding and Generation with Masked Discrete Diffusion
Lijiang Li, Zuwei Long, Yunhang Shen +6
While recent multimodal large language models (MLLMs) have made impressive strides, they predominantly employ a conventional autoregressive architecture as their backbone, leaving…
OmniVideo-100K: A Dataset for Audio-Visual Reasoning through Structured Scripts and Evidence Chains
Xinyue Cai, Chaoyou Fu, Yi-Fan Zhang +2
Current automated pipelines for audio-visual Question Answering (QA) generally adopt a ``video-caption-QA'' paradigm. However, these methods typically segment videos into short cli…
VideoDetective: Clue Hunting via both Extrinsic Query and Intrinsic Relevance for Long Video Understanding
Ruoliu Yang, Chu Wu, Caifeng Shan +2
Long video understanding remains challenging for multimodal large language models (MLLMs) due to limited context windows, which necessitate identifying sparse query-relevant video…