SpEx+: A Complete Time Domain Speaker Extraction Network

Abstract

Speaker extraction aims to extract the target speech signal from a multi-talker environment given a target speaker's reference speech. We recently proposed a time-domain solution, SpEx, that avoids the phase estimation in frequency-domain approaches. Unfortunately, SpEx is not fully a time-domain solution since it performs time-domain speech encoding for speaker extraction, while taking frequency-domain speaker embedding as the reference. The size of the analysis window for time-domain and the size for frequency-domain input are also different. Such mismatch has an adverse effect on the system performance. To eliminate such mismatch, we propose a complete time-domain speaker extraction solution, that is called SpEx+. Specifically, we tie the weights of two identical speech encoder networks, one for the encoder-extractor-decoder pipeline, another as part of the speaker encoder. Experiments show that the SpEx+ achieves 0.8dB and 2.1dB SDR improvement over the state-of-the-art SpEx baseline, under different and same gender conditions on WSJ0-2mix-extr database respectively.

accepted in INTERSPEECH 2020

References in corpus (5)

• Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separation
• Deep attractor network for single-microphone speaker separation
• Speaker-independent Speech Separation with Deep Attractor Network
• VoiceFilter: Targeted Voice Separation by Speaker-Conditioned Spectrogram Masking
• End-to-End Speech Separation with Unfolded Iterative Phase Reconstruction

Cited by in corpus (4)

• Time-Domain Speech Extraction with Spatial Information and Multi Speaker Conditioning Mechanism
• Speaker-conditioned Target Speaker Extraction based on Customized LSTM Cells
• Investigation of Practical Aspects of Single Channel Speech Separation for ASR
• WASE: Learning When to Attend for Speaker Extraction in Cocktail Party Environments