10 citations · 13 across the 22 of their papers we have counts for
25 papers
InstMoE: Adaptive Multimodal Routing with Specialized Experts
Guimin Hu, Xiang He, Yingjian Li +3
Multimodal inputs are inherently heterogeneous, not only across modalities but also in the information pathways required for effective prediction. To address this limitation, we pr…
RobotEQ 3.0: Towards Personalized Social Proactive Intelligence in Embodied Agents
Shufan Zhang, Xinyi Che, Kuofei Fang +8
Social Proactive Intelligence (SPI) is an emerging research area, aiming to shift embodied agents from reactive assistance toward proactively understanding human needs and executin…
RobotEQ-Video: A Video-Centric Benchmark for Social Proactive Intelligence with World-State Taxonomy
Xinyi Che, Zheng Lian, Kuofei Fang +13
Social Proactive Intelligence (SPI) extends proactive assistance beyond task completeness to consider social appropriateness in diverse embodied scenarios. However, prior SPI resea…
OneEmo: A Unified Multimodal Reasoning Model for Emotion Perception, Understanding, and Interaction
Jiahao Huang, Zheng Lian, Jingyi Zhang +3
Multimodal Large Language Models (MLLMs) have demonstrated remarkable capabilities in emotional intelligence. However, prevailing research predominantly focuses on task-specific sp…
AffectSim: A Controllable Interactive 3D Simulation Benchmark for Embodied Affective Perception
Ke Xing, Zhilong Wang, Zheng Lian +8
Existing affective benchmarks largely rely on fixed recordings, where observation conditions are predetermined before inference. Consequently, they mainly evaluate passive affect r…
VideoGAIA: A Benchmark for General AI Assistants on Agentic Video Understanding
Fan Zhang, Guangming Yao, Jinyang Wu +6
Video understanding is a fundamental task for evaluating the capabilities of multimodal large language models (MLLMs). However, existing leading models have already achieved approx…
AC-VLA: Robust Out-of-Distribution Action Execution via Compositional Learning
Xiaojiang Peng, Kai Peng, Jie Lu +3
Vision-Language-Action (VLA) models excel at end-to-end robotic manipulation but struggle with out-of-distribution (OOD) generalization when familiar sub-tasks are recombined in un…
OmniOPSD: Rationale-Privileged On-Policy Self-Distillation for Affective Computing
Zebang Cheng, Shuimu Chen, Boxue Yang +7
Multimodal large language models (MLLMs) have advanced affective computing by enabling nuanced reasoning over visual, acoustic, and linguistic evidence. A key approach to improving…
Struct-Searcher: Agentic Structural Thinking Advances Multimodal Deep Information Seeking
Fan Zhang, Vireo Zhang, Shengju Qian +7
Deep research agents have attracted increasing attention for their ability to collect large-scale online information to acquire target knowledge, with recent efforts shifting from…
AffectGPT-RL: Revealing Roles of Reinforcement Learning in Open-Vocabulary Emotion Recognition
Zheng Lian, Fan Zhang, Lan Chen +8
Open-Vocabulary Multimodal Emotion Recognition (OV-MER) aims to predict emotions without being constrained by predefined label spaces, thereby enabling fine-grained emotion underst…
RobotEQ: Towards Social Proactive Intelligence in Embodied Agents
Kuofei Fang, Zheng Lian, Xinyi Che +13
Embodied agents represent a prominent research focus across both academia and industry. The prevailing paradigm has gradually shifted from reactive assistance, which requires expli…
Emotion-LLaMAv2 and MMEVerse: A New Framework and Benchmark for Multimodal Emotion Understanding
Xiaojiang Peng, Jingyi Chen, Zebang Cheng +11
Understanding human emotions from multimodal signals poses a significant challenge in affective computing and human-robot interaction. While multimodal large language models (MLLMs…
SayNext-Bench: Why Do LLMs Struggle with Next-Utterance Anticipation?
Yueyi Yang, Haotian Liu, Fang Kang +4
We explore the use of large language models (LLMs) for next-utterance anticipation in human dialogue. Despite recent advances in LLMs demonstrating their ability to engage in natur…
Rethinking Facial Expression Recognition in the Era of Multimodal Large Language Models: Benchmark, Datasets, and Beyond
Fan Zhang, Haoxuan Li, Shengju Qian +9
Multimodal Large Language Models (MLLMs) have revolutionized numerous research fields, including computer vision and affective computing. As a pivotal challenge in this interdiscip…
MME-Emotion: A Holistic Evaluation Benchmark for Emotional Intelligence in Multimodal Large Language Models
Fan Zhang, Zebang Cheng, Chong Deng +18
Recent advances in multimodal large language models (MLLMs) have catalyzed transformative progress in affective computing, enabling models to exhibit emergent emotional intelligenc…
AffectGPT-R1: Leveraging Reinforcement Learning for Open-Vocabulary Multimodal Emotion Recognition
Zheng Lian, Fan Zhang, Yazhou Zhang +5
Open-Vocabulary Multimodal Emotion Recognition (OV-MER) aims to predict emotions without being constrained by label spaces, enabling fine-grained emotion understanding. Unlike trad…
Learning Transferable Facial Emotion Representations from Large-Scale Semantically Rich Captions
Licai Sun, Xingxun Jiang, Haoyu Chen +7
Current facial emotion recognition systems are predominately trained to predict a fixed set of predefined categories or abstract dimensional values. This constrained form of superv…
EmoPrefer: Can Large Language Models Understand Human Emotion Preferences?
Zheng Lian, Licai Sun, Lan Chen +8
Descriptive Multimodal Emotion Recognition (DMER) has garnered increasing research attention. Unlike traditional discriminative paradigms that rely on predefined emotion taxonomies…
MMAR: A Challenging Benchmark for Deep Reasoning in Speech, Audio, Music, and Their Mix
Ziyang Ma, Yinghao Ma, Yanqiao Zhu +31
We introduce MMAR, a new benchmark designed to evaluate the deep reasoning capabilities of Audio-Language Models (ALMs) across massive multi-disciplinary tasks. MMAR comprises 1,00…
Emotion-Qwen: A Unified Framework for Emotion and Vision Understanding
Dawei Huang, Qing Li, Chuan Yan +9
Accurate emotion understanding in videos necessitates effectively recognizing and interpreting emotional states by integrating visual, textual, auditory, and contextual cues. Altho…
MER 2025: When Affective Computing Meets Large Language Models
Zheng Lian, Rui Liu, Kele Xu +15
MER2025 is the third year of our MER series of challenges, aiming to bring together researchers in the affective computing community to explore emerging trends and future direction…
EmoBench-M: Benchmarking Emotional Intelligence for Multimodal Large Language Models
He Hu, Lianzhong You, Hongbo Xu +7
With the integration of multimodal large language models (MLLMs) into robotic systems and AI applications, embedding emotional intelligence (EI) capabilities is essential for enabl…
AffectGPT: A New Dataset, Model, and Benchmark for Emotion Understanding with Multimodal Large Language Models
Zheng Lian, Haoyu Chen, Lan Chen +9
The emergence of multimodal large language models (MLLMs) advances multimodal emotion recognition (MER) to the next level, from naive discriminative tasks to complex emotion unders…
OV-MER: Towards Open-Vocabulary Multimodal Emotion Recognition
Zheng Lian, Haiyang Sun, Licai Sun +13
Multimodal Emotion Recognition (MER) is a critical research area that seeks to decode human emotions from diverse data modalities. However, existing machine learning methods predom…