SADDEL: Joint Speech Separation and Denoising Model based on Multitask Learning

Abstract

Speech data collected in real-world scenarios often encounters two issues. First, multiple sources may exist simultaneously, and the number of sources may vary with time. Second, the existence of background noise in recording is inevitable. To handle the first issue, we refer to speech separation approaches, that separate speech from an unknown number of speakers. To address the second issue, we refer to speech denoising approaches, which remove noise components and retrieve pure speech signals. Numerous deep learning based methods for speech separation and denoising have been proposed that show promising results. However, few works attempt to address the issues simultaneously, despite speech separation and denoising tasks having similar nature. In this study, we propose a joint speech separation and denoising framework based on the multitask learning criterion to tackle the two issues simultaneously. The experimental results show that the proposed framework not only performs well on both speech separation and denoising tasks, but also outperforms related methods in most conditions.

arXiv commentThe two first authors made equal contributions

References in corpus (5)

• Speaker-independent Speech Separation with Deep Attractor Network
• Conditional Generative Adversarial Networks for Speech Enhancement and Noise-Robust Speaker Verification
• Deep Clustering and Conventional Networks for Music Separation: Stronger Together
• Improving Universal Sound Separation Using Sound Classification
• Two-stage model and optimal SI-SNR for monaural multi-speaker speech separation in noisy environment