3 citations · 5 across the 16 of their papers we have counts for
17 papers
Learning to Use Imagination: Progress-Conditioned Future Utilization for World Action Models
Yijie Zhu, Zitong Yu, Wei Li +4
World Action Models (WAMs) extend Vision-Language-Action (VLA) models by incorporating future visual dynamics into action generation. However, existing WAMs often utilize imagined…
PCA: Persistence-Aware Compression and Aggregation for Fast Video Large Language Models
Zihan Song, Shuo Ye, Bo Zhao +4
Despite advances in Video Large Language Models (VLLMs) that have displayed promising outcomes in video understanding, the redundancy in the long-duration frames remains a hindranc…
AC-VLA: Robust Out-of-Distribution Action Execution via Compositional Learning
Xiaojiang Peng, Kai Peng, Jie Lu +3
Vision-Language-Action (VLA) models excel at end-to-end robotic manipulation but struggle with out-of-distribution (OOD) generalization when familiar sub-tasks are recombined in un…
GMoT: Gated Motion-Aware Tokenization for Fine-Grained Micro-Gesture Video Reasoning with Multimodal LLMs
Taorui Wang, Wei Xia, Hui Ma +5
Micro-gesture recognition demands the detection of fleeting, spatially localized movements that are frequently overwhelmed by dominant static appearances and background noise. Whil…
RSICCLLM: A Multimodal Large Language Model for Remote Sensing Image Change Captioning
Yelin Wang, Zijia Song, Shuo Ye +6
Remote Sensing Image Change Captioning (RSICC) aims to describe changes between bi-temporal remote sensing images and holds significant research and application value. However, mos…
OracleAnalyser: Analysing Implicit Semantics of Oracle Bone Scripts through MLLMs with Post-training
Zijia Song, Yelin Wang, Zhengyi Ma +5
With the advancement of artificial intelligence, research on oracle bone scripts has entered a new era. However, existing methods and benchmarks remain largely confined to recognit…
AffectVerse: Emotional World Models for Multimodal Affective Computing
Bo Zhao, Fanghua Ye, Yixin Ji +3
Humans infer emotions by integrating observed multimodal cues with expectations about how affective states may unfold. Existing multimodal large language models (MLLMs), however, o…
Navigating the Emotion Tree: Hierarchical Hyperbolic RAG for Multimodal Emotion Recognition
Zeheng Wang, Bo Zhao, Yijie Zhu +6
Multimodal emotion recognition aims to integrate text, audio, and video sources to understand human affective states. Although multimodal large language models excel at multimodal…
AffectAgent: Collaborative Multi-Agent Reasoning for Retrieval-Augmented Multimodal Emotion Recognition
Zeheng Wang, Zitong Yu, Yijie Zhu +9
LLM-based multimodal emotion recognition relies on static parametric memory and often hallucinates when interpreting nuanced affective states. In this paper, given that single-roun…
VLA: Prior-Guided Vision-Language-Action Models via World Knowledge Variation
Yijie Zhu, Jie He, Rui Shao +4
Recent vision-language-action (VLA) models have significantly advanced robotic manipulation by unifying perception, reasoning, and control. To achieve such integration, recent stud…
AULLM++: Structured-Token-Conditioned Large Language Models for Micro-Expression Action Unit Detection
Zhishu Liu, Kaishen Yuan, Bo Zhao +2
Micro-expression Action Unit (AU) detection identifies localized AUs from subtle facial muscle activations, providing a foundation for decoding affective cues. Previous methods fac…
SUGAR: Learning Skeleton Representation with Visual-Motion Knowledge for Action Recognition
Qilang Ye, Yu Zhou, Lian He +10
Large Language Models (LLMs) hold rich implicit knowledge and powerful transferability. In this paper, we explore the combination of LLMs with the human skeleton to perform action…
When Eyes and Ears Disagree: Can MLLMs Discern Audio-Visual Confusion?
Qilang Ye, Wei Zeng, Meng Liu +4
Can Multimodal Large Language Models (MLLMs) discern confused objects that are visually present but audio-absent? To study this, we introduce a new benchmark, AV-ConfuseBench, whic…
AV-Master: Dual-Path Comprehensive Perception Makes Better Audio-Visual Question Answering
Jiayu Zhang, Shuo Ye, Qilang Ye +3
Audio-Visual Question Answering (AVQA) requires models to effectively utilize both visual and auditory modalities to answer complex and diverse questions about audio-visual scenes.…
PGD-Imp: Rethinking and Unleashing Potential of Classic PGD with Dual Strategies for Imperceptible Adversarial Attacks
Jin Li, Zitong Yu, Ziqiang He +2
Imperceptible adversarial attacks have recently attracted increasing research interests. Existing methods typically incorporate external modules or loss terms other than a simple $…
Answering Diverse Questions via Text Attached with Key Audio-Visual Clues
Qilang Ye, Zitong Yu, Xin Liu
Audio-visual question answering (AVQA) requires reference to video content and auditory information, followed by correlating the question to predict the most precise answer. Althou…
CAT: Enhancing Multimodal Large Language Model to Answer Questions in Dynamic Audio-Visual Scenarios
Qilang Ye, Zitong Yu, Rui Shao +3
This paper focuses on the challenge of answering questions in scenarios that are composed of rich and complex dynamic audio-visual components. Although existing Multimodal Large La…