12 papers
What Do Deepfake Audio Detectors Hear? From Self-Supervised Models to Audio LLMs
Wanying Ge, Xin Wang, Junichi Yamagishi
Real-world audio recordings for deepfake detection rarely consist solely of speech; they often are mixtures of speech and background events that themselves may also be synthetic. W…
Textual Acoustic Grounding for Generalizable LLM-Based Deepfake Voice Detection
Yassine El Kheir, Xin Wang, Wanqing Ge +3
Deepfake voice detection suffers from poor generalization across unseen domains. While Audio Large Language Models (ALLMs) show promise, the modality gap between continuous audio e…
A Training-Free Proactive Defense Against Partial Speech Manipulation via Self-Embedding Steganography
Yigitcan Özer, Zhe Zhang, Wanying Ge +2
Partial deepfake speech, where only limited segments of an utterance are synthesized or manipulated, poses a significant challenge to existing deepfake detection systems. As the pr…
Toward Interpretable Speech Deepfake Detection using Artifact-Specific Experts and Calibrated Detection Scores
Viola Negroni, Xin Wang, Wanying Ge +3
In this work, we propose an interpretable framework for speech deepfake detection based on artifact-specific expert models. Rather than relying on black-box decisions, the framewor…
Does Fine-tuning by Reinforcement Learning Improve Generalization in Binary Speech Deepfake Detection?
Xin Wang, Ge Wanying, Junichi Yamagishi
Building speech deepfake detection models that are generalizable to unseen attacks remains a challenging problem. Although the field has shifted toward a pre-training and fine-tuni…
Deepfake Word Detection by Next-token Prediction using Fine-tuned Whisper
Hoan My Tran, Xin Wang, Wanying Ge +2
Deepfake speech utterances can be forged by replacing one or more words in a bona fide utterance with semantically different words synthesized with speech-generative models. While…
Self Voice Conversion as an Attack against Neural Audio Watermarking
Yigitcan Özer, Wanying Ge, Zhe Zhang +2
Audio watermarking embeds auxiliary information into speech while maintaining speaker identity, linguistic content, and perceptual quality. Although recent advances in neural and d…
FakeMark: Deepfake Speech Attribution With Watermarked Artifacts
Wanying Ge, Xin Wang, Junichi Yamagishi
Deepfake speech attribution remains challenging for existing solutions. Classifier-based solutions often fail to generalize to domain-shifted samples, and watermarking-based soluti…
Towards Data Drift Monitoring for Speech Deepfake Detection in the context of MLOps
Xin Wang, Wanying Ge, Junichi Yamagishi
When being delivered in applications or services on the cloud, static speech deepfake detectors that are not updated will become vulnerable to newly created speech deepfake attacks…
LENS-DF: Deepfake Detection and Temporal Localization for Long-Form Noisy Speech
Xuechen Liu, Wanying Ge, Xin Wang +1
This study introduces LENS-DF, a novel and comprehensive recipe for training and evaluating audio deepfake detection and temporal localization under complicated and realistic audio…
A Comparative Study on Proactive and Passive Detection of Deepfake Speech
Chia-Hua Wu, Wanying Ge, Xin Wang +3
Solutions for defending against deepfake speech fall into two categories: proactive watermarking models and passive conventional deepfake detectors. While both address common threa…
Post-training for Deepfake Speech Detection
Wanying Ge, Xin Wang, Xuechen Liu +1
We introduce a post-training approach that adapts self-supervised learning (SSL) models for deepfake speech detection by bridging the gap between general pre-training and domain-sp…