11 papers
PhysAgent: A Multi-Agent Framework for Reliable Remote Heart Rate Estimation
Yehui Yang, Bo Zhao, Junzhe Cao +4
Remote photoplethysmography (rPPG) enables non-contact heart-rate estimation from facial videos, but its weak physiological signal is easily corrupted by motion, illumination chang…
GMoT: Gated Motion-Aware Tokenization for Fine-Grained Micro-Gesture Video Reasoning with Multimodal LLMs
Taorui Wang, Wei Xia, Hui Ma +5
Micro-gesture recognition demands the detection of fleeting, spatially localized movements that are frequently overwhelmed by dominant static appearances and background noise. Whil…
Harmonious Parameter Adaptation in Continual Visual Instruction Tuning for Safety-Aligned MLLMs
Ziqi Wang, Chang Che, Qi Wang +4
While continual visual instruction tuning (CVIT) has shown promise in adapting multimodal large language models (MLLMs), existing studies predominantly focus on models without safe…
StrLoRA: Towards Streaming Continual Visual Instruction Tuning for MLLMs
Chang Che, Ziqi Wang, Hui Ma +2
Continual Visual Instruction Tuning (CVIT) enables Multimodal Large Language Models to incrementally acquire new abilities. However, existing CVIT methods operate under a restricti…
Navigating the Emotion Tree: Hierarchical Hyperbolic RAG for Multimodal Emotion Recognition
Zeheng Wang, Bo Zhao, Yijie Zhu +6
Multimodal emotion recognition aims to integrate text, audio, and video sources to understand human affective states. Although multimodal large language models excel at multimodal…
LoRA in LoRA: Towards Parameter-Efficient Architecture Expansion for Continual Visual Instruction Tuning
Chang Che, Ziqi Wang, Pengwan Yang +3
Continual Visual Instruction Tuning (CVIT) enables Multimodal Large Language Models (MLLMs) to incrementally learn new tasks over time. However, this process is challenged by catas…