Noise Adaptive Speech Enhancement using Domain Adversarial Training

Abstract

In this study, we propose a novel noise adaptive speech enhancement (SE) system, which employs a domain adversarial training (DAT) approach to tackle the issue of a noise type mismatch between the training and testing conditions. Such a mismatch is a critical problem in deep-learning-based SE systems. A large mismatch may cause a serious performance degradation to the SE performance. Because we generally use a well-trained SE system to handle various unseen noise types, a noise type mismatch commonly occurs in real-world scenarios. The proposed noise adaptive SE system contains an encoder-decoder-based enhancement model and a domain discriminator model. During adaptation, the DAT approach encourages the encoder to produce noise-invariant features based on the information from the discriminator model and consequentially increases the robustness of the enhancement model to unseen noise types. Herein, we regard stationary noises as the source domain (with the ground truth of clean speech) and non-stationary noises as the target domain (without the ground truth). We evaluated the proposed system on TIMIT sentences. The experiment results show that the proposed noise adaptive SE system successfully provides significant improvements in PESQ (19.0%), SSNR (39.3%), and STOI (27.0%) over the SE system without an adaptation.

arXiv commentAccepted to Interspeech 2019

References in corpus (12)

• Adam: A Method for Stochastic Optimization
• Unsupervised Domain Adaptation with Residual Transfer Networks
• A DIRT-T Approach to Unsupervised Domain Adaptation
• Conditional Generative Adversarial Networks for Speech Enhancement and Noise-Robust Speaker Verification
• Tacotron: Towards End-to-End Speech Synthesis
• Speaker-Invariant Training via Adversarial Learning
• Many Paths to Equilibrium: GANs Do Not Need to Decrease a Divergence At Every Step
• Unsupervised Adaptation with Domain Separation Networks for Robust Speech Recognition
• Invariant Representations for Noisy Speech Recognition
• Cycle-Consistent Speech Enhancement
• Adversarial Feature-Mapping for Speech Enhancement
• Domain Adversarial Training for Accented Speech Recognition

Cited by in corpus (6)

• Accent-Robust Automatic Speech Recognition Using Supervised and Unsupervised Wav2vec Embeddings
• Unsupervised Noise Adaptive Speech Enhancement by Discriminator-Constrained Optimal Transport
• A Two-stage Complex Network using Cycle-consistent Generative Adversarial Networks for Speech Enhancement
• Incorporating Symbolic Sequential Modeling for Speech Enhancement
• Dynamic Layer Customization for Noise Robust Speech Emotion Recognition in Heterogeneous Condition Training
• Enhancing Low-Quality Voice Recordings Using Disentangled Channel Factor and Neural Waveform Model