On Loss Functions for Supervised Monaural Time-Domain Speech Enhancement

Abstract

Many deep learning-based speech enhancement algorithms are designed to minimize the mean-square error (MSE) in some transform domain between a predicted and a target speech signal. However, optimizing for MSE does not necessarily guarantee high speech quality or intelligibility, which is the ultimate goal of many speech enhancement algorithms. Additionally, only little is known about the impact of the loss function on the emerging class of time-domain deep learning-based speech enhancement systems. We study how popular loss functions influence the performance of deep learning-based speech enhancement systems. First, we demonstrate that perceptually inspired loss functions might be advantageous if the receiver is the human auditory system. Furthermore, we show that the learning rate is a crucial design parameter even for adaptive gradient-based optimizers, which has been generally overlooked in the literature. Also, we found that waveform matching performance metrics must be used with caution as they in certain situations can fail completely. Finally, we show that a loss function based on scale-invariant signal-to-distortion ratio (SI-SDR) achieves good general performance across a range of popular speech enhancement evaluation metrics, which suggests that SI-SDR is a good candidate as a general-purpose loss function for speech enhancement systems.

arXiv commentPublished in the IEEE Transactions on Audio, Speech and Language Processing

References in corpus (2)

• Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separation
• On the Relationship Between Short-Time Objective Intelligibility and Short-Time Spectral-Amplitude Mean-Square Error for Speech Enhancement

Cited by in corpus (6)

• On The Compensation Between Magnitude and Phase in Speech Separation
• WaveCRN: An Efficient Convolutional Recurrent Neural Network for End-to-end Speech Enhancement
• A Neural-Network Framework for the Design of Individualised Hearing-Loss Compensation
• End-to-End Complex-Valued Multidilated Convolutional Neural Network for Joint Acoustic Echo Cancellation and Noise Suppression
• Learning to Maximize Speech Quality Directly Using MOS Prediction for Neural Text-to-Speech
• Perceive and predict: self-supervised speech representation based loss functions for speech enhancement