Time-Domain Speech Extraction with Spatial Information and Multi Speaker Conditioning Mechanism
arXiv:2102.03762 · doi:10.1109/ICASSP39728.2021.9414092
Abstract
In this paper, we present a novel multi-channel speech extraction system to simultaneously extract multiple clean individual sources from a mixture in noisy and reverberant environments. The proposed method is built on an improved multi-channel time-domain speech separation network which employs speaker embeddings to identify and extract multiple targets without label permutation ambiguity. To efficiently inform the speaker information to the extraction model, we propose a new speaker conditioning mechanism by designing an additional speaker branch for receiving external speaker embeddings. Experiments on 2-channel WHAMR! data show that the proposed system improves by 9% relative the source separation performance over a strong multi-channel baseline, and it increases the speech recognition accuracy by more than 16% relative over the same baseline.
arXiv commentAccepted for ICASSP 2021
References in corpus (13)
- Adam: A Method for Stochastic Optimization
- Instance Normalization: The Missing Ingredient for Fast Stylization
- VoxCeleb2: Deep Speaker Recognition
- Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separation
- Pyroomacoustics: A Python package for audio room simulations and array processing algorithms
- Voice Separation with an Unknown Number of Multiple Speakers
- Sudo rm -rf: Efficient Networks for Universal Audio Source Separation
- On End-to-end Multi-channel Time Domain Speech Separation in Reverberant Environments
- SpEx+: A Complete Time Domain Speaker Extraction Network
- Improving speaker discrimination of target speech extraction with time-domain SpeakerBeam
- Speaker-Conditional Chain Model for Speech Separation and Extraction
- Enhancing End-to-End Multi-channel Speech Separation via Spatial Feature Learning
- Interrupted and cascaded permutation invariant training for speech separation