Phase-aware Speech Enhancement with Deep Complex U-Net

Abstract

Most deep learning-based models for speech enhancement have mainly focused on estimating the magnitude of spectrogram while reusing the phase from noisy speech for reconstruction. This is due to the difficulty of estimating the phase of clean speech. To improve speech enhancement performance, we tackle the phase estimation problem in three ways. First, we propose Deep Complex U-Net, an advanced U-Net structured model incorporating well-defined complex-valued building blocks to deal with complex-valued spectrograms. Second, we propose a polar coordinate-wise complex-valued masking method to reflect the distribution of complex ideal ratio masks. Third, we define a novel loss function, weighted source-to-distortion ratio (wSDR) loss, which is designed to directly correlate with a quantitative evaluation measure. Our model was evaluated on a mixture of the Voice Bank corpus and DEMAND database, which has been widely used by many deep learning models for speech enhancement. Ablation experiments were conducted on the mixed dataset showing that all three proposed approaches are empirically valid. Experimental results show that the proposed method achieves state-of-the-art performance in all metrics, outperforming previous approaches by a large margin.

Significant error was found in data processing step, therefore will be retracted from International Conference on Learning Representations (ICLR) 2019. It is not recommended to read current version

Cited by in corpus (19)

• End-to-End Multi-Channel Speech Separation
• DCCRN: Deep Complex Convolution Recurrent Network for Phase-Aware Speech Enhancement
• Speech Denoising Without Clean Training Data: A Noise2Noise Approach
• Improving Perceptual Quality by Phone-Fortified Perceptual Loss using Wasserstein Distance for Speech Enhancement
• Dynamic Noise Embedding: Noise Aware Training and Adaptation for Speech Enhancement
• TSTNN: Two-stage Transformer based Neural Network for Speech Enhancement in the Time Domain
• Temporal-Spatial Neural Filter: Direction Informed End-to-End Multi-channel Target Speech Separation
• Real-time Denoising and Dereverberation with Tiny Recurrent U-Net
• DCCRN+: Channel-wise Subband DCCRN with SNR Estimation for Speech Enhancement
• Phase-aware Single-stage Speech Denoising and Dereverberation with U-Net
• RHR-Net: A Residual Hourglass Recurrent Neural Network for Speech Enhancement
• Time-domain Speech Enhancement with Generative Adversarial Learning
• Data Efficient Voice Cloning from Noisy Samples with Domain Adversarial Training
• Monaural Speech Enhancement with Complex Convolutional Block Attention Module and Joint Time Frequency Losses
• Improved Speech Separation with Time-and-Frequency Cross-domain Joint Embedding and Clustering
• End-to-end speech enhancement based on discrete cosine transform
• Real-time Streaming Wave-U-Net with Temporal Convolutions for Multichannel Speech Enhancement
• Neural Networks with Complex-Valued Weights Have No Spurious Local Minima
• Coarse-to-fine Optimization for Speech Enhancement