128 citations · 382 across the 72 of their papers we have counts for
75 papers
Word-Aligned Prosody Reflects Speaker Differences More Than Depression or PTSD Severity
Hsiang-Chen Yeh, Aurosweta Mahapatra, Shreeram Suresh Chandra +2
Word-aligned prosody is increasingly used to detect depression and posttraumatic stress disorder (PTSD), yet the stability and clinical relevance of these measurements remain uncle…
Cross-Lingual F5-TTS 2: A Simplified Framework for Language-Agnostic Voice Cloning
Qingyu Liu, Rixi Xu, Yushen Chen +11
Zero-shot text-to-speech (TTS) can clone a speaker's voice from a short audio prompt, yet most TTS systems still require the audio prompt transcript during inference. This dependen…
Robust Cross-Domain Speech-Based Alzheimer's Disease Detection via Iterative Adversarial Self-Training
Luqi Sun, Shreeram Suresh Chandra, Aurosweta Mahapatra +3
As Alzheimer's disease (AD) has increasingly become a major global public health issue, speech-based AD detection has attracted widespread attention. However, most existing methods…
Not All Attacks Are Learned Equally in Speech Deepfake Detection
Avantika Singh, Aurosweta Mahapatra, Ismail Rasim Ulgen +3
Speech deepfake detection (SDD) models are trained on multi-attack datasets containing diverse spoofing systems, such as text-to-speech (TTS) and voice conversion (VC). In standard…
Beyond Accuracy: ARIA-Rubrics for Evaluating Audio Reasoning in Large Audio Language Models
Yupei Li, Qiyang Sun, Mohamed Mady +4
Large Audio Language Models (LALMs) have shown strong performance on audio reasoning benchmarks, but accuracy alone cannot distinguish true reasoning from superficial pattern match…
Brain2Speech-Net: Fast and Intelligible Brain-to-Speech Synthesis Without Text Decoding
Shreeram Suresh Chandra, Zexin Cai, Yu Tsao +2
The loss of speech limits communication for individuals with paralysis. Direct neural-to-speech synthesis is challenging due to the limited availability of neural data for training…
Perceptible or Not? Diagnosing Passive Fingerprints for Speech Deepfake Attribution
Yupei Li, Qiyang Sun, Emmanouil Benetos +2
Passive fingerprints (intrinsic traces naturally left by generators) have been shown to enable attribution in speech deepfake detection, yet their persistence, reproducibility, and…
Cleaner Speech, Weaker Generalization: Revisiting Pitt-Derived Benchmarks for Alzheimer's Disease Detection
Luqi Sun, Shreeram Suresh Chandra, Lin Zhang +5
Speech-based Alzheimer's disease (AD) detection increasingly relies on speech-enhanced and curated versions of the Pitt Corpus, where speech enhancement, sample selection, and demo…
Is Prosody Lost in Translation? Fine-Grained Cross-Lingual Prosody Similarity Across Languages
Haopeng Xie, Ismail Rasim Ulgen, Sofia Son +2
Prosody plays an important role in speech translation, conveying information such as emphasis, emotion, and intent beyond lexical content. However, despite recent progress in expre…
Emotion Across Speech and Faces: Shared Affective Mechanisms in Multimodal Foundation Models
Xiutian Zhao, Luqi Sun, Björn Schuller +1
Modern multimodal foundation models (MFMs) have made rapid progress on tasks requiring integrated perception across speech, vision, and language, including emotion recognition. How…
Multilingual Emotion Neurons in Large Audio-Language Models
Xiutian Zhao, Philipp Koehn, Björn Schuller +1
Emotion is central to human communication, and its expression varies across languages. Large audio-language models (LALMs) achieve strong performance on multilingual speech tasks,…
AffectDF: The Most Comprehensive Benchmark for Speech Deepfake Detection against Emotionally Expressive Attacks
Aurosweta Mahapatra, Xiutian Zhao, Shreeram Suresh Chandra +7
Speech deepfake detection (SDD) systems achieve strong performance on conventional benchmarks; however, existing datasets provide limited coverage of emotionally expressive and rec…
Disentangling the Interpretive and Predictive Roles of LIWC: Controlled Substitution in Depression-Related Classification
Hsiang-Chen Yeh, Xiutian Zhao, Aurosweta Mahapatra +3
Linguistic Inquiry and Word Count (LIWC) provides auditable psycholinguistic categories that are widely used to interpret depression-related language, but its incremental predictiv…
TRACE-EVC: Text-Guided Relative Affective Control for Zero-Shot Emotional Voice Conversion
Zihan Zhang, Shreeram Suresh Chandra, Zongyang Du +5
Traditional emotional voice conversion (EVC) conditions generation on explicit target emotions like labels or references, defining the target affective state but omitting the direc…
XAI-Grounded Explanation Generation for Speech Deepfake Detection with Training-Free Multimodal Large Language Models
Yupei Li, Qiyang Sun, Xiaoliang Wu +3
Speech deepfake detection (SDD) systems require trustworthy explanations for reliable decision-making. Existing explanation ways mainly fall into two categories. Traditional explai…
X-Voice: Enabling Everyone to Speak 30 Languages via Zero-Shot Cross-Lingual Voice Cloning
Rixi Xu, Qingyu Liu, Haitao Li +10
In this paper, we present X-Voice, a 0.4B multilingual zero-shot voice cloning model that clones arbitrary voices and enables everyone to speak 30 languages. X-Voice is trained on…
Who is Speaking or Who is Depressed? A Controlled Study of Speaker Leakage in Speech-Based Depression Detection
Hsiang-Chen Yeh, Luqi Sun, Aurosweta Mahapatra +3
This study investigates whether speech-based depression detection models learn depression-related acoustic biomarkers or instead rely on speaker identity cues. Using the DAIC-WOZ d…
ProSDD: Learning Prosodic Representations for Speech Deepfake Detection against Expressive and Emotional Attacks
Aurosweta Mahapatra, Ismail Rasim Ulgen, Kong Aik Lee +2
Speech deepfake detection (SDD) systems perform well on standard benchmarks datasets but often fail to generalize to expressive and emotional spoofing attacks. Many methods rely on…
DiffAnon: Diffusion-based Prosody Control for Voice Anonymization
Ismail Rasim Ulgen, Zexin Cai, Nicholas Andrews +2
To preserve or not to preserve prosody is a central question in voice anonymization. Prosody conveys meaning and affect, yet is tightly coupled with speaker identity. Existing meth…
Neuron-Level Emotion Control in Speech-Generative Large Audio-Language Models
Xiutian Zhao, Ismail Rasim Ulgen, Philipp Koehn +2
Large audio-language models (LALMs) can produce expressive speech, yet reliable emotion control remains elusive: conversions often miss the target affect and may degrade linguistic…
Universal Speech Content Factorization
Henry Li Xinyuan, Zexin Cai, Lin Zhang +5
We propose Universal Speech Content Factorization (USCF), a simple and invertible linear method for extracting a low-rank speech representation in which speaker timbre is suppresse…
Discovering and Causally Validating Emotion-Sensitive Neurons in Large Audio-Language Models
Xiutian Zhao, Björn Schuller, Berrak Sisman
Emotion is a central dimension of spoken communication, yet, we still lack a mechanistic account of how modern large audio-language models (LALMs) encode it internally. We present…
NaturalVoices: A Large-Scale, Spontaneous and Emotional Podcast Dataset for Voice Conversion
Zongyang Du, Shreeram Suresh Chandra, Ismail Rasim Ulgen +4
Everyday speech conveys far more than words, it reflects who we are, how we feel, and the circumstances surrounding our interactions. Yet, most existing speech datasets are acted,…
HuLA: Prosody-Aware Anti-Spoofing with Multi-Task Learning for Expressive and Emotional Synthetic Speech
Aurosweta Mahapatra, Ismail Rasim Ulgen, Berrak Sisman
Current anti-spoofing systems remain vulnerable to expressive and emotional synthetic speech, since they rarely leverage prosody as a discriminative cue. Prosody is central to huma…