28 citations · 57 across the 30 of their papers we have counts for
30 papers
AT-ADD: A Benchmark and Challenge for Robust and All-Type Audio Deepfake Detection
Yuankun Xie, Haonan Cheng, Jiayi Zhou +12
Recent audio generation models can synthesize high-fidelity speech, environmental sound, singing voice, and music, creating new risks for multimedia trust. Existing audio deepfake…
AT-ADD: All-Type Audio Deepfake Detection Challenge Summary
Yuankun Xie, Haonan Cheng, Jiayi Zhou +11
This paper summarizes the ACM Multimedia 2026 AT-ADD Grand Challenge on all-type audio deepfake detection. AT-ADD contains two tracks: robust speech deepfake detection under realis…
AT-ADD: All-Type Audio Deepfake Detection Challenge Evaluation Plan
Yuankun Xie, Haonan Cheng, Jiayi Zhou +10
The rapid advancement of Audio Large Language Models (ALLMs) has enabled cost-effective, high-fidelity generation and manipulation of both speech and non-speech audio, including so…
Edit Content, Preserve Acoustics: Imperceptible Text-Based Speech Editing via Self-Consistency Rewards
Yong Ren, Jiangyan Yi, Jianhua Tao +3
Imperceptible text-based speech editing modifies spoken content through transcript manipulation while preserving acoustic continuity. Prior acoustic-space approaches suffer from co…
Interpretable All-Type Audio Deepfake Detection with Audio LLMs via Frequency-Time Reinforcement Learning
Yuankun Xie, Xiaoxuan Guo, Jiayi Zhou +6
Recent advances in audio large language models (ALLMs) have made high-quality synthetic audio widely accessible, increasing the risk of malicious audio deepfakes across speech, env…
T2AV-Compass: Towards Unified Evaluation for Text-to-Audio-Video Generation
Zhe Cao, Tao Wang, Jiaming Wang +10
Text-to-Audio-Video (T2AV) generation aims to synthesize temporally coherent video and semantically synchronized audio from natural language, yet its evaluation remains fragmented,…
SecoustiCodec: Cross-Modal Aligned Streaming Single-Codecbook Speech Codec
Chunyu Qiang, Haoyu Wang, Cheng Gong +10
Speech codecs serve as a crucial bridge in unifying speech and text language models. Existing codec methods face several challenges in semantic encoding, such as residual paralingu…
P2Mark: Plug-and-play Parameter-level Watermarking for Neural Speech Generation
Yong Ren, Jiangyan Yi, Tao Wang +7
Neural speech generation (NSG) has rapidly advanced as a key component of artificial intelligence-generated content, enabling the generation of high-quality, highly realistic speec…
Mel-Refine: A Plug-and-Play Approach to Refine Mel-Spectrogram in Audio Generation
Hongming Guo, Ruibo Fu, Yizhong Geng +9
Text-to-audio (TTA) model is capable of generating diverse audio from textual prompts. However, most mainstream TTA models, which predominantly rely on Mel-spectrograms, still face…
DPI-TTS: Directional Patch Interaction for Fast-Converging and Style Temporal Modeling in Text-to-Speech
Xin Qi, Ruibo Fu, Zhengqi Wen +12
In recent years, speech diffusion models have advanced rapidly. Alongside the widely used U-Net architecture, transformer-based models such as the Diffusion Transformer (DiT) have…
Text Prompt is Not Enough: Sound Event Enhanced Prompt Adapter for Target Style Audio Generation
Chenxu Xiong, Ruibo Fu, Shuchen Shi +9
Current mainstream audio generation methods primarily rely on simple text prompts, often failing to capture the nuanced details necessary for multi-style audio generation. To addre…
WMCodec: End-to-End Neural Speech Codec with Deep Watermarking for Authenticity Verification
Junzuo Zhou, Jiangyan Yi, Yong Ren +3
Recent advances in speech spoofing necessitate stronger verification mechanisms in neural speech codecs to ensure authenticity. Current methods embed numerical watermarks before co…
VQ-CTAP: Cross-Modal Fine-Grained Sequence Representation Learning for Speech Processing
Chunyu Qiang, Wang Geng, Yi Zhao +12
Deep learning has brought significant improvements to the field of cross-modal representation learning. For tasks such as text-to-speech (TTS), voice conversion (VC), and automatic…
ICAGC 2024: Inspirational and Convincing Audio Generation Challenge 2024
Ruibo Fu, Rui Liu, Chunyu Qiang +11
The Inspirational and Convincing Audio Generation Challenge 2024 (ICAGC 2024) is part of the ISCSLP 2024 Competitions and Challenges track. While current text-to-speech (TTS) techn…
ASRRL-TTS: Agile Speaker Representation Reinforcement Learning for Text-to-Speech Speaker Adaptation
Ruibo Fu, Xin Qi, Zhengqi Wen +10
Speaker adaptation, which involves cloning voices from unseen speakers in the Text-to-Speech task, has garnered significant interest due to its numerous applications in multi-media…
MINT: a Multi-modal Image and Narrative Text Dubbing Dataset for Foley Audio Content Planning and Generation
Ruibo Fu, Shuchen Shi, Hongming Guo +12
Foley audio, critical for enhancing the immersive experience in multimedia content, faces significant challenges in the AI-generated content (AIGC) landscape. Despite advancements…
PPPR: Portable Plug-in Prompt Refiner for Text to Audio Generation
Shuchen Shi, Ruibo Fu, Zhengqi Wen +10
Text-to-Audio (TTA) aims to generate audio that corresponds to the given text description, playing a crucial role in media production. The text descriptions in TTA datasets lack ri…
TraceableSpeech: Towards Proactively Traceable Text-to-Speech with Watermarking
Junzuo Zhou, Jiangyan Yi, Tao Wang +5
Various threats posed by the progress in text-to-speech (TTS) have prompted the need to reliably trace synthesized speech. However, contemporary approaches to this task involve add…
Anything in Any Scene: Photorealistic Video Object Insertion
Chen Bai, Zeman Shao, Guoxiang Zhang +11
Realistic video simulation has shown significant potential across diverse applications, from virtual reality to film production. This is particularly true for scenarios where captu…
Fewer-token Neural Speech Codec with Time-invariant Codes
Yong Ren, Tao Wang, Jiangyan Yi +4
Language model based text-to-speech (TTS) models, like VALL-E, have gained attention for their outstanding in-context learning capability in zero-shot scenarios. Neural speech code…
Residual Speaker Representation for One-Shot Voice Conversion
Le Xu, Jiangyan Yi, Tao Wang +4
Recently, there have been significant advancements in voice conversion, resulting in high-quality performance. However, there are still two critical challenges in this field. First…
Learning Speech Representation From Contrastive Token-Acoustic Pretraining
Chunyu Qiang, Hao Li, Yixin Tian +4
For fine-grained generation and recognition tasks such as minimally-supervised text-to-speech (TTS), voice conversion (VC), and automatic speech recognition (ASR), the intermediate…
Boosting Fast and High-Quality Speech Synthesis with Linear Diffusion
Haogeng Liu, Tao Wang, Jie Cao +2
Denoising Diffusion Probabilistic Models have shown extraordinary ability on various generative tasks. However, their slow inference speed renders them impractical in speech synthe…
ADD 2023: the Second Audio Deepfake Detection Challenge
Jiangyan Yi, Jianhua Tao, Ruibo Fu +15
Audio deepfake detection is an emerging topic in the artificial intelligence community. The second Audio Deepfake Detection Challenge (ADD 2023) aims to spur researchers around the…