1 citations · 1 across the 24 of their papers we have counts for
25 papers
CARES: A Controlled Synthetic Benchmark of Speaker Reactions to Sound
Marcel Gibier, Thomas Thebaud, Olivier Boëffard +1
Automatic audio scene description turns a recording into a text account of a situation. One difficulty is deciding which elements of the audio should be kept, since a description c…
Who Are They to Each Other? Multi-Agent Reasoning for Speaker Relationship Inference
Yaohan Guan, Yen-Ju Lu, Yuzhe Wang +5
Inferring speaker relationships from spoken conversations is an important step towards socially aware speech understanding. However, this task remains underexplored, and supervised…
When Text Misleads: Inconsistent-Aware Reasoning for Audio-Grounded Dialogue
Yen-Ju Lu, Yuzhe Wang, Yaohan Guan +8
Understanding spoken dialogue requires joint reasoning over lexical content and paralinguistic acoustic signals such as emotion and conversational intent. However, existing evaluat…
Leveraging Gradient Reversal Loss and Multitask Learning for Datasets-Aware Audio Deepfake Detection
Mingrui Liang, Thomas Thebaud, Lukasz Wojciak +4
Recent advances in speech synthesis and voice conversion, which pose threats to security and privacy, have underscored the need for deepfake detection technology. Although existing…
SPEARBench: A Benchmark for Naturalness Evaluation in Streaming Speech-to-Speech Language Models
Thomas Thebaud, Yuzhe Wang, Hao Zhang +5
Streaming speech-to-speech language models aim to answer spoken queries directly with synthetic speech. However, standard speech and text benchmarks do not capture whether these sy…
ProPS: Prompted Profile Synthesis for Natural Language-Conditioned Speaker Embedding Distributions
Thomas Thebaud, Junhyeok Lee, Laureano Moro-Velazquez +2
Speaker embeddings, or x-vectors, are widely used to represent speaker identity and speaker-related attributes, but existing embedding extractors are typically descriptive rather t…
TurnNat: Automatic Evaluation of Turn-Taking Naturalness in Dyadic Spoken Dialogue
Hao Zhang, Thomas Thebaud, Georgi Tinchev +2
Turn-taking naturalness is central to full-duplex spoken dialogue systems, yet its automatic evaluation remains limited. Existing evaluations often rely on human judgments or behav…
TRACE: Temporal Relationship-Aware Conversational Entrainment Detection in Dyadic Speech
Sathvik Manikantan Napa Ugandhar, Hao Zhang, Alison Gunzler +5
With the proliferation of speech AI agents, understanding emotional entrainment in conversational interaction has become increasingly important. Emotional entrainment is shaped by…
Reference-Based Prosody and Rhythm Evaluation for Spoken Dialogue Systems
Ashish Hallur, Thomas Thebaud, Georgi Tinchev +2
Speech-to-speech (S2S) AI agents are advancing rapidly, yet evaluation lacks interpretable speech-native measures for conversational prosody and rhythm. Because , speaking rat…
StanceBench: A Benchmark for Audio LLM-Based Interpersonal Stance Evaluation from Speech
Yuzhe Wang, Thomas Thebaud, Jennifer Hu +5
Speech-to-speech dialogue models increasingly depend on prosody and interactional nuance to convey social intent, yet benchmarks for these cues remain limited. We introduce StanceB…
DiT-Flow: Speech Enhancement Robust to Multiple Distortions based on Flow Matching in Latent Space and Diffusion Transformers
Tianyu Cao, Helin Wang, Ari Frummer +7
Recent advances in generative models, such as diffusion and flow matching, have shown strong performance in audio tasks. However, speech enhancement (SE) models are typically train…
Can LLMs Help Localize Fake Words in Partially Fake Speech?
Lin Zhang, Thomas Thebaud, Zexin Cai +5
Large language models (LLMs), trained on large-scale text, have recently attracted significant attention for their strong performance across many tasks. Motivated by this, we inves…
Speaker Verification with Speech-Aware LLMs: Evaluation and Augmentation
Thomas Thebaud, Yuzhe Wang, Laureano Moro-Velazquez +2
Speech-aware large language models (LLMs) can accept speech inputs, yet their training objectives largely emphasize linguistic content or specific fields such as emotions or the sp…
Reconstruct! Don't Encode: Self-Supervised Representation Reconstruction Loss for High-Intelligibility and Low-Latency Streaming Neural Audio Codec
Junhyeok Lee, Xiluo He, Jihwan Lee +6
Neural audio codecs optimized for mel-spectrogram reconstruction often fail to preserve intelligibility. While semantic encoder distillation improves encoded representations, it do…
ReFESS-QI: Reference-Free Evaluation For Speech Separation With Joint Quality And Intelligibility Scoring
Ari Frummer, Helin Wang, Tianyu Cao +6
Source separation is a crucial pre-processing step for various speech processing tasks, such as automatic speech recognition (ASR). Traditionally, the evaluation metrics for speech…
Where Do Backdoors Live? A Component-Level Analysis of Backdoor Propagation in Speech Language Models
Alexandrine Fortier, Thomas Thebaud, Jesús Villalba +3
Speech language models (SLMs) are systems of systems: independent components that unite to achieve a common goal. Despite their heterogeneous nature, SLMs are often studied end-to-…
Rhythm Features for Speaker Identification
Nick Mehlman, Thomas Thebaud, Dani Byrd +1
While deep learning models have demonstrated robust performance in speaker recognition tasks, they primarily rely on low-level audio features learned empirically from spectrograms…
Vox-Profile: A Speech Foundation Model Benchmark for Characterizing Diverse Speaker Traits and Speech Properties
Tiantian Feng, Jihwan Lee, Anfeng Xu +9
We introduce Vox-Profile, a comprehensive benchmark to characterize rich speaker traits and speech properties using speech foundation models. Unlike prior work that targets a singl…
SoloSpeech: Enhancing Intelligibility and Quality in Target Speech Extraction through a Cascaded Generative Pipeline
Helin Wang, Jiarui Hai, Dongchao Yang +7
Target Speech Extraction (TSE) aims to isolate a target speaker's voice from a mixture of multiple speakers by leveraging speaker-specific cues, typically provided as auxiliary aud…
Demographic Attributes Prediction from Speech Using WavLM Embeddings
Yuchen Yang, Thomas Thebaud, Najim Dehak
This paper introduces a general classifier based on WavLM features, to infer demographic characteristics, such as age, gender, native language, education, and country, from speech.…
Detecting Neurodegenerative Diseases using Frame-Level Handwriting Embeddings
Sarah Laouedj, Yuzhe Wang, Jesus Villalba +3
In this study, we explored the use of spectrograms to represent handwriting signals for assessing neurodegenerative diseases, including 42 healthy controls (CTL), 35 subjects with…
Clean Label Attacks against SLU Systems
Henry Li Xinyuan, Sonal Joshi, Thomas Thebaud +3
Poisoning backdoor attacks involve an adversary manipulating the training data to induce certain behaviors in the victim model by inserting a trigger in the signal at inference tim…
Interpretable Features for the Assessment of Neurodegenerative Diseases through Handwriting Analysis
Thomas Thebaud, Anna Favaro, Casey Chen +4
Motor dysfunction is a common sign of neurodegenerative diseases (NDs) such as Parkinson's disease (PD) and Alzheimer's disease (AD), but may be difficult to detect, especially in…
Supervised and Unsupervised Alignments for Spoofing Behavioral Biometrics
Thomas Thebaud, Gaël Le Lan, Anthony Larcher
Biometric recognition systems are security systems based on intrinsic properties of their users, usually encoded in high dimension representations called embeddings, which potentia…