7 citations · 18 across the 31 of their papers we have counts for
35 papers
What Do Deepfake Audio Detectors Hear? From Self-Supervised Models to Audio LLMs
Wanying Ge, Xin Wang, Junichi Yamagishi
Real-world audio recordings for deepfake detection rarely consist solely of speech; they often are mixtures of speech and background events that themselves may also be synthetic. W…
VoxTubeS: Distributable Speaker-Anonymized Synthetic Speech Corpora and Their Analysis
Zhe Zhang, Yexin Lu, Junichi Yamagishi
Large speech corpora support research, but recordings can expose speaker identity because voice remains a recognizable biometric. Meanwhile, speech data derived from media can be d…
Textual Acoustic Grounding for Generalizable LLM-Based Deepfake Voice Detection
Yassine El Kheir, Xin Wang, Wanqing Ge +3
Deepfake voice detection suffers from poor generalization across unseen domains. While Audio Large Language Models (ALLMs) show promise, the modality gap between continuous audio e…
Combining Self-Embedding Audio Watermarking with Ultra-Low-Bitrate Neural Codecs
Yigitcan Özer, Xin Wang, Zhe Zhang +1
Partial manipulation of speech recordings, where only localized segments of an utterance are altered, poses a significant challenge for content integrity verification, as reliable…
A Training-Free Proactive Defense Against Partial Speech Manipulation via Self-Embedding Steganography
Yigitcan Özer, Zhe Zhang, Wanying Ge +2
Partial deepfake speech, where only limited segments of an utterance are synthesized or manipulated, poses a significant challenge to existing deepfake detection systems. As the pr…
Toward Interpretable Speech Deepfake Detection using Artifact-Specific Experts and Calibrated Detection Scores
Viola Negroni, Xin Wang, Wanying Ge +3
In this work, we propose an interpretable framework for speech deepfake detection based on artifact-specific expert models. Rather than relying on black-box decisions, the framewor…
Synthetic Audio Generation Framework for Air Traffic Control Speech Recognition
Raphaël Bagat, Zhe Zhang, Junichi Yamagishi +2
Automatic Speech Recognition (ASR) systems, despite achieving remarkable accuracy in general-purpose domains with native speech (L1), struggle in domains like Air Traffic Control (…
Does Fine-tuning by Reinforcement Learning Improve Generalization in Binary Speech Deepfake Detection?
Xin Wang, Ge Wanying, Junichi Yamagishi
Building speech deepfake detection models that are generalizable to unseen attacks remains a challenging problem. Although the field has shifted toward a pre-training and fine-tuni…
The Third VoicePrivacy Challenge: Preserving Emotional Expressiveness and Linguistic Content in Voice Anonymization
Natalia Tomashenko, Xiaoxiao Miao, Pierre Champion +7
We present results and analyses from the third VoicePrivacy Challenge held in 2024, which focuses on advancing voice anonymization technologies. The task was to develop a voice ano…
Self Voice Conversion as an Attack against Neural Audio Watermarking
Yigitcan Özer, Wanying Ge, Zhe Zhang +2
Audio watermarking embeds auxiliary information into speech while maintaining speaker identity, linguistic content, and perceptual quality. Although recent advances in neural and d…
ASVspoof 5: Evaluation of Spoofing, Deepfake, and Adversarial Attack Detection Using Crowdsourced Speech
Xin Wang, Héctor Delgado, Nicholas Evans +8
ASVspoof 5 is the fifth edition in a series of challenges which promote the study of speech spoofing and deepfake detection solutions. A significant change from previous challenge…
Human perception of audio deepfakes: the role of language and speaking style
Eugenia San Segundo, Aurora López-Jareño, Xin Wang +1
Audio deepfakes have reached a level of realism that makes it increasingly difficult to distinguish between human and artificial voices, which poses risks such as identity theft or…
FakeMark: Deepfake Speech Attribution With Watermarked Artifacts
Wanying Ge, Xin Wang, Junichi Yamagishi
Deepfake speech attribution remains challenging for existing solutions. Classifier-based solutions often fail to generalize to domain-shifted samples, and watermarking-based soluti…
Target speaker anonymization in multi-speaker recordings
Natalia Tomashenko, Junichi Yamagishi, Xin Wang +2
Most of the existing speaker anonymization research has focused on single-speaker audio, leading to the development of techniques and evaluation metrics optimized for such conditio…
Towards Data Drift Monitoring for Speech Deepfake Detection in the context of MLOps
Xin Wang, Wanying Ge, Junichi Yamagishi
When being delivered in applications or services on the cloud, static speech deepfake detectors that are not updated will become vulnerable to newly created speech deepfake attacks…
Can External Sources Help the Knowledge Cut-off Issues in Audio Deepfake Detection?
Xuechen Liu, Xin Wang, Junichi Yamagishi +1
In conventional deepfake audio detection pipelines, inference and evaluation are typically conducted using standalone, parameterized models trained exclusively on a fixed dataset.…
LENS-DF: Deepfake Detection and Temporal Localization for Long-Form Noisy Speech
Xuechen Liu, Wanying Ge, Xin Wang +1
This study introduces LENS-DF, a novel and comprehensive recipe for training and evaluating audio deepfake detection and temporal localization under complicated and realistic audio…
MIDI-VALLE: Improving Expressive Piano Performance Synthesis Through Neural Codec Language Modelling
Jingjing Tang, Xin Wang, Zhe Zhang +3
Generating expressive audio performances from music scores requires models to capture both instrument acoustics and human interpretation. Traditional music performance synthesis pi…
Mitigating Language Mismatch in SSL-Based Speaker Anonymization
Zhe Zhang, Wen-Chin Huang, Xin Wang +2
Speaker anonymization aims to protect speaker identity while preserving content information and the intelligibility of speech. However, most speaker anonymization systems (SASs) ar…
From Sharpness to Better Generalization for Speech Deepfake Detection
Wen Huang, Xuechen Liu, Xin Wang +2
Generalization remains a critical challenge in speech deepfake detection (SDD). While various approaches aim to improve robustness, generalization is typically assessed through per…
Post-training for Deepfake Speech Detection
Wanying Ge, Xin Wang, Xuechen Liu +1
We introduce a post-training approach that adapts self-supervised learning (SSL) models for deepfake speech detection by bridging the gap between general pre-training and domain-sp…
ASVspoof 5: Design, Collection and Validation of Resources for Spoofing, Deepfake, and Adversarial Attack Detection Using Crowdsourced Speech
Xin Wang, Héctor Delgado, Hemlata Tak +26
ASVspoof 5 is the fifth edition in a series of challenges which promote the study of speech spoofing and deepfake attacks as well as the design of detection solutions. We introduce…
Towards An Integrated Approach for Expressive Piano Performance Synthesis from Music Scores
Jingjing Tang, Erica Cooper, Xin Wang +2
This paper presents an integrated system that transforms symbolic music scores into expressive piano performance audio. By combining a Transformer-based Expressive Performance Rend…
A Preliminary Case Study on Long-Form In-the-Wild Audio Spoofing Detection
Xuechen Liu, Xin Wang, Junichi Yamagishi
Audio spoofing detection has become increasingly important due to the rise in real-world cases. Current spoofing detectors, referred to as spoofing countermeasures (CM), are mainly…