23 citations · 47 across the 24 of their papers we have counts for
24 papers
OneEmo: A Unified Multimodal Reasoning Model for Emotion Perception, Understanding, and Interaction
Jiahao Huang, Zheng Lian, Jingyi Zhang +3
Multimodal Large Language Models (MLLMs) have demonstrated remarkable capabilities in emotional intelligence. However, prevailing research predominantly focuses on task-specific sp…
Robust Speech Emotion Recognition under Tone-Word Conflict: A Benchmark and Framework
Xiaojiang Peng, Dawei Huang, Yongjie Lv +5
Speech emotion recognition (SER) is a crucial component of human-computer interaction, attracting extensive attention from both industry and academia. However, existing SER systems…
AffectSim: A Controllable Interactive 3D Simulation Benchmark for Embodied Affective Perception
Ke Xing, Zhilong Wang, Zheng Lian +8
Existing affective benchmarks largely rely on fixed recordings, where observation conditions are predetermined before inference. Consequently, they mainly evaluate passive affect r…
HiMA-MDD: A Hierarchical Multi-Agent Harness for Interpretable Multimodal Depression Detection in Clinical Interviews
Ao Chen, Xiaojiang Peng
Depression assessment from multimodal clinical interviews requires integrating dispersed evidence from multiple symptoms into a coherent PHQ-8 profile. This process is hierarchical…
What Can I Edit? Open-Ended Strategy Discovery and the Emotion Editability Landscape
Qing Li, Zeyu Dong, Yin Cui +2
Emotional image editing requires more than applying affective filters or modifying predefined visual factors: an effective edit must identify what a particular image can afford for…
AffectVerse: Emotional World Models for Multimodal Affective Computing
Bo Zhao, Fanghua Ye, Yixin Ji +3
Humans infer emotions by integrating observed multimodal cues with expectations about how affective states may unfold. Existing multimodal large language models (MLLMs), however, o…
Emotion-LLaMAv2 and MMEVerse: A New Framework and Benchmark for Multimodal Emotion Understanding
Xiaojiang Peng, Jingyi Chen, Zebang Cheng +11
Understanding human emotions from multimodal signals poses a significant challenge in affective computing and human-robot interaction. While multimodal large language models (MLLMs…
When Tone and Words Disagree: Towards Robust Speech Emotion Recognition under Acoustic-Semantic Conflict
Dawei Huang, Yongjie Lv, Ruijie Xiong +2
Speech Emotion Recognition (SER) systems often assume congruence between vocal emotion and lexical semantics. However, in real-world interactions, acoustic-semantic conflict is com…
MME-Emotion: A Holistic Evaluation Benchmark for Emotional Intelligence in Multimodal Large Language Models
Fan Zhang, Zebang Cheng, Chong Deng +18
Recent advances in multimodal large language models (MLLMs) have catalyzed transformative progress in affective computing, enabling models to exhibit emergent emotional intelligenc…
Emotion-Qwen: A Unified Framework for Emotion and Vision Understanding
Dawei Huang, Qing Li, Chuan Yan +9
Accurate emotion understanding in videos necessitates effectively recognizing and interpreting emotional states by integrating visual, textual, auditory, and contextual cues. Altho…
MER 2025: When Affective Computing Meets Large Language Models
Zheng Lian, Rui Liu, Kele Xu +15
MER2025 is the third year of our MER series of challenges, aiming to bring together researchers in the affective computing community to explore emerging trends and future direction…
Why We Feel: Breaking Boundaries in Emotional Reasoning with Multimodal Large Language Models
Yuxiang Lin, Jingdong Sun, Zhi-Qi Cheng +7
Most existing emotion analysis emphasizes which emotion arises (e.g., happy, sad, angry) but neglects the deeper why. We propose Emotion Interpretation (EI), focusing on causal fac…
AffectGPT: A New Dataset, Model, and Benchmark for Emotion Understanding with Multimodal Large Language Models
Zheng Lian, Haoyu Chen, Lan Chen +9
The emergence of multimodal large language models (MLLMs) advances multimodal emotion recognition (MER) to the next level, from naive discriminative tasks to complex emotion unders…
Mamba-Enhanced Text-Audio-Video Alignment Network for Emotion Recognition in Conversations
Xinran Li, Xiaomao Fan, Qingyang Wu +2
Emotion Recognition in Conversations (ERCs) is a vital area within multimodal interaction research, dedicated to accurately identifying and classifying the emotions expressed by sp…
DPDEdit: Detail-Preserved Diffusion Models for Multimodal Fashion Image Editing
Xiaolong Wang, Zhi-Qi Cheng, Jue Wang +1
Fashion image editing is a crucial tool for designers to convey their creative ideas by visualizing design concepts interactively. Current fashion image editing techniques, though…
Multimodal Multi-turn Conversation Stance Detection: A Challenge Dataset and Effective Model
Fuqiang Niu, Zebang Cheng, Xianghua Fu +5
Stance detection, which aims to identify public opinion towards specific targets using social media data, is an important yet challenging task. With the proliferation of diverse mu…
SZTU-CMU at MER2024: Improving Emotion-LLaMA with Conv-Attention for Multimodal Emotion Recognition
Zebang Cheng, Shuyuan Tu, Dawei Huang +4
This paper presents our winning approach for the MER-NOISE and MER-OV tracks of the MER2024 Challenge on multimodal emotion recognition. Our system leverages the advanced emotional…
FlexEdit: Marrying Free-Shape Masks to VLLM for Flexible Image Editing
Tianshuo Yuan, Yuxiang Lin, Jue Wang +5
Combining Vision Large Language Models (VLLMs) with diffusion models offers a powerful method for executing image editing tasks based on human language instructions. However, langu…
DSMix: Distortion-Induced Sensitivity Map Based Pre-training for No-Reference Image Quality Assessment
Jinsong Shi, Pan Gao, Xiaojiang Peng +1
Image quality assessment (IQA) has long been a fundamental challenge in image understanding. In recent years, deep learning-based IQA methods have shown promising performance. Howe…
Emotion-LLaMA: Multimodal Emotion Recognition and Reasoning with Instruction Tuning
Zebang Cheng, Zhi-Qi Cheng, Jun-Yan He +6
Accurate emotion perception is crucial for various applications, including human-computer interaction, education, and counseling. However, traditional single-modality approaches of…
MIPS at SemEval-2024 Task 3: Multimodal Emotion-Cause Pair Extraction in Conversations with Multimodal Language Models
Zebang Cheng, Fuqiang Niu, Yuxiang Lin +3
This paper presents our winning submission to Subtask 2 of SemEval 2024 Task 3 on multimodal emotion cause analysis in conversations. We propose a novel Multimodal Emotion Recognit…
UMETTS: A Unified Framework for Emotional Text-to-Speech Synthesis with Multimodal Prompts
Zhi-Qi Cheng, Xiang Li, Jun-Yan He +4
Emotional Text-to-Speech (E-TTS) synthesis has garnered significant attention in recent years due to its potential to revolutionize human-computer interaction. However, current E-T…
A Challenge Dataset and Effective Models for Conversational Stance Detection
Fuqiang Niu, Min Yang, Ang Li +3
Previous stance detection studies typically concentrate on evaluating stances within individual instances, thereby exhibiting limitations in effectively modeling multi-party discus…
Invisible Gas Detection: An RGB-Thermal Cross Attention Network and A New Benchmark
Jue Wang, Yuxiang Lin, Qi Zhao +4
The widespread use of various chemical gases in industrial processes necessitates effective measures to prevent their leakage during transportation and storage, given their high to…