11 citations · 12 across the 24 of their papers we have counts for
26 papers
WeSep: A Modular and Cue-Composable Framework for Target Speaker Extraction
Ke Zhang, Xiaoyang Yu, Haoyu Li +3
The study of Target Speaker Extraction (TSE) aims to isolate a desired speaker from overlapping speech mixture given auxiliary cues. Existing systems are typically designed for spe…
SLT 2026 REAL-TSE Challenge: Real-world Target Speaker Extraction from Conversational Recordings
Shuai Wang, Zihan Qian, Ke Zhang +9
We introduce the REAL-TSE Challenge, an IEEE SLT 2026 satellite challenge on target speaker extraction~(TSE) from real conversational recordings. Given a multi-speaker mixture and…
AlphaFlowTSE: One-Step Generative Target Speaker Extraction via Conditional AlphaFlow
Duojia Li, Shuhan Zhang, Zihan Qian +5
In target speaker extraction (TSE), we aim to recover target speech from a multi-talker mixture using a short enrollment utterance as reference. Recent studies on diffusion and flo…
Detect, Attend and Extract: Keyword Guided Target Speaker Extraction
Haoyu Li, Yu Xi, Yidi Jiang +5
Target speaker extraction (TSE) aims to extract the speech of a target speaker from mixtures containing multiple competing speakers. Conventional TSE systems predominantly rely on…
Direct Preference Optimization for Speech Autoregressive Diffusion Models
Zhijun Liu, Dongya Jia, Xiaoqiang Wang +4
Autoregressive diffusion models (ARDMs) have recently been applied to speech generation, achieving state-of-the-art (SOTA) performance in zero-shot text-to-speech. By autoregressiv…
Accent Normalization Using Self-Supervised Discrete Tokens with Non-Parallel Data
Qibing Bai, Sho Inoue, Shuai Wang +3
Accent normalization converts foreign-accented speech into native-like speech while preserving speaker identity. We propose a novel pipeline using self-supervised discrete tokens a…
Multi-Step Prediction and Control of Hierarchical Emotion Distribution in Text-to-Speech Synthesis
Sho Inoue, Kun Zhou, Shuai Wang +1
We investigate hierarchical emotion distribution (ED) for achieving multi-level quantitative control of emotion rendering in text-to-speech synthesis (TTS). We introduce a novel mu…
SpeechRefiner: Towards Perceptual Quality Refinement for Front-End Algorithms
Sirui Li, Shuai Wang, Zhijun Liu +3
Speech pre-processing techniques such as denoising, de-reverberation, and separation, are commonly employed as front-ends for various downstream speech processing tasks. However, t…
SongBloom: Coherent Song Generation via Interleaved Autoregressive Sketching and Diffusion Refinement
Chenyu Yang, Shuai Wang, Hangting Chen +3
Generating music with coherent structure, harmonious instrumental and vocal elements remains a significant challenge in song generation. Existing language models and diffusion-base…
PersonaTAB: Predicting Personality Traits using Textual, Acoustic, and Behavioral Cues in Fully-Duplex Speech Dialogs
Sho Inoue, Shai Wang, Haizhou Li
Despite significant progress in neural spoken dialog systems, personality-aware conversation agents -- capable of adapting behavior based on personalities -- remain underexplored d…
Causal Self-supervised Pretrained Frontend with Predictive Code for Speech Separation
Wupeng Wang, Zexu Pan, Xinke Li +2
Speech separation (SS) seeks to disentangle a multi-talker speech mixture into single-talker speech streams. Although SS can be generally achieved using offline methods, such a pro…
Context-Aware Two-Step Training Scheme for Domain Invariant Speech Separation
Wupeng Wang, Zexu Pan, Jingru Lin +2
Speech separation seeks to isolate individual speech signals from a multi-talk speech mixture. Despite much progress, a system well-trained on synthetic data often experiences perf…
SongEditor: Adapting Zero-Shot Song Generation Language Model as a Multi-Task Editor
Chenyu Yang, Shuai Wang, Hangting Chen +7
The emergence of novel generative modeling paradigms, particularly audio language models, has significantly advanced the field of song generation. Although state-of-the-art models…
Hierarchical Control of Emotion Rendering in Speech Synthesis
Sho Inoue, Kun Zhou, Shuai Wang +1
Emotional text-to-speech synthesis (TTS) aims to generate realistic emotional speech from input text. However, quantitatively controlling multi-level emotion rendering remains chal…
MoMuSE: Momentum Multi-modal Target Speaker Extraction for Real-time Scenarios with Impaired Visual Cues
Junjie Li, Ke Zhang, Shuai Wang +3
Audio-visual Target Speaker Extraction (AV-TSE) aims to isolate the speech of a specific target speaker from an audio mixture using time-synchronized visual cues. In real-world sce…
Speech Separation with Pretrained Frontend to Minimize Domain Mismatch
Wupeng Wang, Zexu Pan, Xinke Li +2
Speech separation seeks to separate individual speech signals from a speech mixture. Typically, most separation models are trained on synthetic data due to the unavailability of ta…
VLMimic: Vision Language Models are Visual Imitation Learner for Fine-grained Actions
Guanyan Chen, Meiling Wang, Te Cui +9
Visual imitation learning (VIL) provides an efficient and intuitive strategy for robotic systems to acquire novel skills. Recent advancements in Vision Language Models (VLMs) have…
Multi-Level Speaker Representation for Target Speaker Extraction
Ke Zhang, Junjie Li, Shuai Wang +4
Target speaker extraction (TSE) relies on a reference cue of the target to extract the target speech from a speech mixture. While a speaker embedding is commonly used as the refere…
WeSep: A Scalable and Flexible Toolkit Towards Generalizable Target Speaker Extraction
Shuai Wang, Ke Zhang, Shaoxiong Lin +6
Target speaker extraction (TSE) focuses on isolating the speech of a specific target speaker from overlapped multi-talker speech, which is a typical setup in the cocktail party pro…
M-Vec: Matryoshka Speaker Embeddings with Flexible Dimensions
Shuai Wang, Pengcheng Zhu, Haizhou Li
Fixed-dimensional speaker embeddings have become the dominant approach in speaker modeling, typically spanning hundreds to thousands of dimensions. These dimensions are hyperparame…
On the effectiveness of enrollment speech augmentation for Target Speaker Extraction
Junjie Li, Ke Zhang, Shuai Wang +3
Deep learning technologies have significantly advanced the performance of target speaker extraction (TSE) tasks. To enhance the generalization and robustness of these algorithms wh…
E1 TTS: Simple and Fast Non-Autoregressive TTS
Zhijun Liu, Shuai Wang, Pengcheng Zhu +2
This paper introduces Easy One-Step Text-to-Speech (E1 TTS), an efficient non-autoregressive zero-shot text-to-speech system based on denoising diffusion pretraining and distributi…
MacST: Multi-Accent Speech Synthesis via Text Transliteration for Accent Conversion
Sho Inoue, Shuai Wang, Wanxing Wang +3
In accented voice conversion or accent conversion, we seek to convert the accent in speech from one another while preserving speaker identity and semantic content. In this study, w…
Overview of Speaker Modeling and Its Applications: From the Lens of Deep Speaker Representation Learning
Shuai Wang, Zhengyang Chen, Kong Aik Lee +2
Speaker individuality information is among the most critical elements within speech signals. By thoroughly and accurately modeling this information, it can be utilized in various i…