Investigating Generative Adversarial Networks based Speech Dereverberation for Robust Speech Recognition

Abstract

We investigate the use of generative adversarial networks (GANs) in speech dereverberation for robust speech recognition. GANs have been recently studied for speech enhancement to remove additive noises, but there still lacks of a work to examine their ability in speech dereverberation and the advantages of using GANs have not been fully established. In this paper, we provide deep investigations in the use of GAN-based dereverberation front-end in ASR. First, we study the effectiveness of different dereverberation networks (the generator in GAN) and find that LSTM leads a significant improvement as compared with feed-forward DNN and CNN in our dataset. Second, further adding residual connections in the deep LSTMs can boost the performance as well. Finally, we find that, for the success of GAN, it is important to update the generator and the discriminator using the same mini-batch data during training. Moreover, using reverberant spectrogram as a condition to discriminator, as suggested in previous studies, may degrade the performance. In summary, our GAN-based dereverberation front-end achieves 14%-19% relative CER reduction as compared to the baseline DNN dereverberation network when tested on a strong multi-condition training acoustic model.

arXiv commentInterspeech 2018

References in corpus (7)

• Adam: A Method for Stochastic Optimization
• Conditional Generative Adversarial Nets
• The Microsoft 2017 Conversational Speech Recognition System
• Entropy-based Pruning of Backoff Language Models
• Conditional Generative Adversarial Networks for Speech Enhancement and Noise-Robust Speaker Verification
• Batch Renormalization: Towards Reducing Minibatch Dependence in Batch-Normalized Models
• Speech Dereverberation Using Nonnegative Convolutive Transfer Function and Spectro temporal Modeling

Cited by in corpus (8)

• Libri-Adapt: A New Speech Dataset for Unsupervised Domain Adaptation
• iSEGAN: Improved Speech Enhancement Generative Adversarial Networks
• Adversarial Joint Training with Self-Attention Mechanism for Robust End-to-End Speech Recognition
• Late reverberation suppression using U-nets
• Efficient acoustic feature transformation in mismatched environments using a Guided-GAN
• Dereverberation using joint estimation of dry speech signal and acoustic system
• NPU Speaker Verification System for INTERSPEECH 2020 Far-Field Speaker Verification Challenge
• Investigation of Monaural Front-End Processing for Robust ASR without Retraining or Joint-Training