Improving Perceptual Quality by Phone-Fortified Perceptual Loss using Wasserstein Distance for Speech Enhancement
arXiv:2010.15174
Abstract
Speech enhancement (SE) aims to improve speech quality and intelligibility, which are both related to a smooth transition in speech segments that may carry linguistic information, e.g. phones and syllables. In this study, we propose a novel phone-fortified perceptual loss (PFPL) that takes phonetic information into account for training SE models. To effectively incorporate the phonetic information, the PFPL is computed based on latent representations of the wav2vec model, a powerful self-supervised encoder that renders rich phonetic information. To more accurately measure the distribution distances of the latent representations, the PFPL adopts the Wasserstein distance as the distance measure. Our experimental results first reveal that the PFPL is more correlated with the perceptual evaluation metrics, as compared to signal-level losses. Moreover, the results showed that the PFPL can enable a deep complex U-Net SE model to achieve highly competitive performance in terms of standardized quality and intelligibility evaluations on the Voice Bank-DEMAND dataset.
References in corpus (8)
- Instance Normalization: The Missing Ingredient for Fast Stylization
- Beyond One-hot Encoding: lower dimensional target embedding
- MetricGAN: Generative Adversarial Networks based Black-box Metric Scores Optimization for Speech Enhancement
- Learning with Learned Loss Function: Speech Enhancement with Quality-Net to Improve Perceptual Evaluation of Speech Quality
- Phase-aware Speech Enhancement with Deep Complex U-Net
- DCCRN: Deep Complex Convolution Recurrent Network for Phase-Aware Speech Enhancement
- Monaural Speech Enhancement using Deep Neural Networks by Maximizing a Short-Time Objective Intelligibility Measure
- End-to-End Multi-Task Denoising for joint SDR and PESQ Optimization