13 papers
Time-Frequency Consistency Learning for Robust Speech Deepfake Detection
Jun Xue, Zhuolin Yi, Yanzhen Ren +6
Recently, speech deepfake detection (SDD) has achieved significant progress. However, its robustness evaluation remains largely confined to controlled additive noise scenarios, lac…
Exploring the Scale and Diversity of Speech Anti-spoofing Datasets: Experiments and Analysis
Zhuolin Yi, Jun Xue, Yanzhen Ren +5
The scale of speech anti-spoofing datasets has grown exponentially over the past decade, driven by the assumption that larger data leads to better performance. However, it remains…
Profiling the Voice: Speaker-Specific Phoneme Fingerprinting for Speech Deepfake Detection
Jun Xue, Tong Zhang, Zhuolin Yi +4
The rapid advancement of generative AI has made audio deepfakes increasingly indistinguishable from authentic human vocals, posing significant threats to persons-of-interest (POI)…
RTCFake: Speech Deepfake Detection in Real-Time Communication
Jun Xue, Zhuolin Yi, Yihuan Huang +6
With the rapid advancement of speech generation technologies, the threat posed by speech deepfakes in real-time communication (RTC) scenarios has intensified. However, existing det…
When AVSR Meets Video Conferencing: Dataset, Degradation, and the Hidden Mechanism Behind Performance Collapse
Yihuan Huang, Jun Xue, Liu Jiajun +5
Audio-Visual Speech Recognition (AVSR) has achieved remarkable progress in offline conditions, yet its robustness in real-world video conferencing (VC) remains largely unexplored.…
ML-ITW: A Multilingual in-the-wild Benchmark for Speech Deepfake Detection
Daixian Li, Jun Xue, Yanzhen Ren +4
Recent advances in speech synthesis and voice conversion have greatly improved the naturalness and authenticity of generated audio. Meanwhile, evolving encoding, compression, and t…
Unifying Speech Editing Detection and Content Localization via Prior-Enhanced Audio LLMs
Jun Xue, Yi Chai, Yanzhen Ren +6
Existing speech editing detection (SED) datasets are predominantly constructed using manual splicing or limited editing operations, resulting in restricted diversity and poor cover…
EchoFake: A Replay-Aware Dataset for Practical Speech Deepfake Detection
Tong Zhang, Yihuan Huang, Yanzhen Ren
The growing prevalence of speech deepfakes has raised serious concerns, particularly in real-world scenarios such as telephone fraud and identity theft. While many anti-spoofing sy…
Backdoor Samples Detection Based on Perturbation Discrepancy Consistency in Pre-trained Language Models
Zuquan Peng, Jianming Fu, Lixin Zou +3
The use of unvetted third-party and internet data renders pre-trained models susceptible to backdoor attacks. Detecting backdoor samples is critical to prevent backdoor activation…
Audio-visual Event Localization on Portrait Mode Short Videos
Wuyang Liu, Yi Chai, Yongpeng Yan +1
Audio-visual event localization (AVEL) plays a critical role in multimodal scene understanding. While existing datasets for AVEL predominantly comprise landscape-oriented long vide…
PASE: Phoneme-Aware Speech Encoder to Improve Lip Sync Accuracy for Talking Head Synthesis
Yihuan Huang, Jiajun Liu, Yanzhen Ren +3
Recent talking head synthesis works typically adopt speech features extracted from large-scale pre-trained acoustic models. However, the intrinsic many-to-many relationship between…
FA-GAN: Artifacts-free and Phase-aware High-fidelity GAN-based Vocoder
Rubing Shen, Yanzhen Ren, Zongkun Sun
Generative adversarial network (GAN) based vocoders have achieved significant attention in speech synthesis with high quality and fast inference speed. However, there still exist m…
Semantic Proximity Alignment: Towards Human Perception-consistent Audio Tagging by Aligning with Label Text Description
Wuyang Liu, Yanzhen Ren
Most audio tagging models are trained with one-hot labels as supervised information. However, one-hot labels treat all sound events equally, ignoring the semantic hierarchy and pro…