4 papers
VAPO: End-to-end Slide-Enhanced Speech Recognition with Omni-modal Large Language Models
Rui Hu, Delai Qiu, Yining Wang +2
Omni-modal large language models (OLLMs) offer a promising end-to-end solution for slide-enhanced speech recognition due to their inherent multimodal capabilities. However, we foun…
CAVALRY-V: A Large-Scale Generator Framework for Adversarial Attacks on Video MLLMs
Jiaming Zhang, Rui Hu, Qing Guo +1
Video Multimodal Large Language Models (V-MLLMs) have shown impressive capabilities in temporal reasoning and cross-modal understanding, yet their vulnerability to adversarial atta…
Investigating and Enhancing Vision-Audio Capability in Omnimodal Large Language Models
Rui Hu, Delai Qiu, Shuyu Wei +4
Omnimodal Large Language Models (OLLMs) have shown significant progress in integrating vision and text, but still struggle with integrating vision and audio, often exhibiting subop…
ODE: Open-Set Evaluation of Hallucinations in Multimodal Large Language Models
Yahan Tu, Rui Hu, Jitao Sang
Hallucination poses a persistent challenge for multimodal large language models (MLLMs). However, existing benchmarks for evaluating hallucinations are generally static, which may…