11 papers
CapRiCorn-1K: A Comprehensive Benchmark for Video Captioning and Subject Referential Consistency Across Temporal Scales
Xinlong Chen, Jiafu Tang, Yue Ding +12
Accurate and comprehensive video captions with consistent subject references are critical for downstream understanding and generation tasks. However, few existing benchmarks can ob…
OmniSIFT: Modality-Asymmetric Token Compression for Efficient Omni-modal Large Language Models
Yue Ding, Yiyan Ji, Jungang Li +12
Omni-modal Large Language Models (Omni-LLMs) have demonstrated strong capabilities in audio-video understanding tasks. However, their reliance on long multimodal token sequences le…
Unifying Language-Action Understanding and Generation for Autonomous Driving
Xinyang Wang, Qian Liu, Wenjie Ding +7
Vision-Language-Action (VLA) models are emerging as a promising paradigm for end-to-end autonomous driving, valued for their potential to leverage world knowledge and reason about…
Multimodal Adaptive Retrieval Augmented Generation through Internal Representation Learning
Ruoshuang Du, Xin Sun, Qiang Liu +4
Visual Question Answering systems face reliability issues due to hallucinations, where models generate answers misaligned with visual input or factual knowledge. While Retrieval Au…
DiaDem: Advancing Dialogue Descriptions in Audiovisual Video Captioning for Multimodal Large Language Models
Xinlong Chen, Weihong Lin, Jingyun Hua +10
Accurate dialogue description in audiovisual video captioning is crucial for downstream understanding and generation tasks. However, existing models generally struggle to produce f…
AVoCaDO: An Audiovisual Video Captioner Driven by Temporal Orchestration
Xinlong Chen, Yue Ding, Weihong Lin +9
Audiovisual video captioning aims to generate semantically rich descriptions with temporal alignment between visual and auditory events, thereby benefiting both video understanding…