10 papers
Causal-EVC: Breaking Emotional Spurious Causality via Spatiotemporal Grounding and Counterfactual Intervention
Cheng Ye, Weidong Chen, Peipei Song +1
Emotional Video Captioning aims to generate factually accurate and emotionally empathetic descriptions. While recent methods have recognized the importance of visual causes to guid…
Multi-Faceted Evaluation and Mitigation of Emotion Hallucinations in MLLMs
Bowen Zeng, Peipei Song, Weidong Chen +5
Multimodal large language models (MLLMs) have shown strong potential in open-ended emotion understanding, yet they often generate emotion hallucinations. Evaluating such hallucinat…
ProFocus: Interpreting Affective Experience in Artistic Images with Progressive Visual Focusing
Zhiyan Zhang, Zicheng Yan, Jianqi Chen +3
Interpreting the emotional responses triggered by images is central to achieving emotional intelligence. Compared with natural images, visual art is intentionally created to elicit…
MMHBench: A Multi-Perspective Benchmark for Mental Health Understanding in Long-Form Videos
Jinpeng Hu, Erqiang Wang, Shan Wang +4
Mental health understanding in long-form videos requires nuanced reasoning over observable behavior, interpersonal context, and latent psychological states. Existing benchmarks lar…
Omni-Perception Policy Optimization for Multimodal Emotion Reasoning
Zhiyuan Han, Beier Zhu, Wenwen Tong +6
We find that current emotion-oriented Omni-MLLMs still lack reliable omni-modal perception: they (i) underutilize multimodal cues in their reasoning trajectories and (ii) exhibit u…
Benchmarking Dynamic Affective Reasoning: A Viewer-Centric Video Emotion Dataset
Zhiyan Zhang, Peipei Song, Jinpeng Hu +3
Video emotion analysis is typically framed as a static classification problem, treating each clip as an independent labeled unit. However, such a formulation overlooks a key psycho…
A Multi-Agent Framework with Structured Reasoning and Reflective Refinement for Multimodal Empathetic Response Generation
Liping Wang, Cheng Ye, Weidong Chen +3
Multimodal empathetic response generation (MERG) aims to generate emotionally engaging and empathetic responses based on users' multimodal contexts. Existing approaches usually rel…
FACE-net: Factual Calibration and Emotion Augmentation for Retrieval-enhanced Emotional Video Captioning
Weidong Chen, Cheng Ye, Zhendong Mao +5
Emotional Video Captioning (EVC) is an emerging task, which aims to describe factual content with the intrinsic emotions expressed in videos. Existing works perceive global emotion…
Beyond Emotion Recognition: A Multi-Turn Multimodal Emotion Understanding and Reasoning Benchmark
Jinpeng Hu, Hongchang Shi, Chongyuan Dai +3
Multimodal large language models (MLLMs) have been widely applied across various fields due to their powerful perceptual and reasoning capabilities. In the realm of psychology, the…
Benchmarking and Bridging Emotion Conflicts for Multimodal Emotion Reasoning
Zhiyuan Han, Beier Zhu, Yanlong Xu +2
Despite their strong performance in multimodal emotion reasoning, existing Multimodal Large Language Models (MLLMs) often overlook the scenarios involving emotion conflicts, where…