4 papers
Enhancing Multimodal Continual Instruction Tuning with BranchLoRA
Duzhen Zhang, Yong Ren, Zhong-Zhi Li +5
Multimodal Continual Instruction Tuning (MCIT) aims to finetune Multimodal Large Language Models (MLLMs) to continually align with human intent across sequential tasks. Existing ap…
Mitigating Audiovisual Mismatch in Visual-Guide Audio Captioning
Le Xu, Chenxing Li, Yong Ren +5
Current vision-guided audio captioning systems frequently fail to address audiovisual misalignment in real-world scenarios, such as dubbed content or off-screen sounds. To bridge t…
Hearing from Silence: Reasoning Audio Descriptions from Silent Videos via Vision-Language Model
Yong Ren, Chenxing Li, Le Xu +7
Humans can intuitively infer sounds from silent videos, but whether multimodal large language models can perform modal-mismatch reasoning without accessing target modalities remain…
Information-Theoretic Complementary Prompts for Improved Continual Text Classification
Duzhen Zhang, Yong Ren, Chenxing Li +2
Continual Text Classification (CTC) aims to continuously classify new text data over time while minimizing catastrophic forgetting of previously acquired knowledge. However, existi…