23 papers
Emotion Across Speech and Faces: Shared Affective Mechanisms in Multimodal Foundation Models
Xiutian Zhao, Luqi Sun, Björn Schuller +1
Modern multimodal foundation models (MFMs) have made rapid progress on tasks requiring integrated perception across speech, vision, and language, including emotion recognition. How…
Multilingual Emotion Neurons in Large Audio-Language Models
Xiutian Zhao, Philipp Koehn, Björn Schuller +1
Emotion is central to human communication, and its expression varies across languages. Large audio-language models (LALMs) achieve strong performance on multilingual speech tasks,…
AffectDF: The Most Comprehensive Benchmark for Speech Deepfake Detection against Emotionally Expressive Attacks
Aurosweta Mahapatra, Xiutian Zhao, Shreeram Suresh Chandra +7
Speech deepfake detection (SDD) systems achieve strong performance on conventional benchmarks; however, existing datasets provide limited coverage of emotionally expressive and rec…
Disentangling the Interpretive and Predictive Roles of LIWC: Controlled Substitution in Depression-Related Classification
Hsiang-Chen Yeh, Xiutian Zhao, Aurosweta Mahapatra +3
Linguistic Inquiry and Word Count (LIWC) provides auditable psycholinguistic categories that are widely used to interpret depression-related language, but its incremental predictiv…
TRACE-EVC: Text-Guided Relative Affective Control for Zero-Shot Emotional Voice Conversion
Zihan Zhang, Shreeram Suresh Chandra, Zongyang Du +5
Traditional emotional voice conversion (EVC) conditions generation on explicit target emotions like labels or references, defining the target affective state but omitting the direc…
XAI-Grounded Explanation Generation for Speech Deepfake Detection with Training-Free Multimodal Large Language Models
Yupei Li, Qiyang Sun, Xiaoliang Wu +3
Speech deepfake detection (SDD) systems require trustworthy explanations for reliable decision-making. Existing explanation ways mainly fall into two categories. Traditional explai…