Exploring the Best Loss Function for DNN-Based Low-latency Speech Enhancement with Temporal Convolutional Networks

Abstract

Recently, deep neural networks (DNNs) have been successfully used for speech enhancement, and DNN-based speech enhancement is becoming an attractive research area. While time-frequency masking based on the short-time Fourier transform (STFT) has been widely used for DNN-based speech enhancement over the last years, time domain methods such as the time-domain audio separation network (TasNet) have also been proposed. The most suitable method depends on the scale of the dataset and the type of task. In this paper, we explore the best speech enhancement algorithm on two different datasets. We propose a STFT-based method and a loss function using problem-agnostic speech encoder (PASE) features to improve subjective quality for the smaller dataset. Our proposed methods are effective on the Voice Bank + DEMAND dataset and compare favorably to other state-of-the-art methods. We also implement a low-latency version of TasNet, which we submitted to the DNS Challenge and made public by open-sourcing it. Our model achieves excellent performance on the DNS Challenge dataset.

References in corpus (6)

• Adam: A Method for Stochastic Optimization
• An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling
• Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separation
• The INTERSPEECH 2020 Deep Noise Suppression Challenge: Datasets, Subjective Speech Quality and Testing Framework
• Multi-Domain Processing via Hybrid Denoising Networks for Speech Enhancement
• End-to-End Multi-Task Denoising for the Joint Optimization of Perceptual Speech Metrics

Cited by in corpus (13)

• Phase-Aware Deep Speech Enhancement: It's All About The Frame Length
• Investigating the Design Space of Diffusion Models for Speech Enhancement
• Assessing the Generalization Gap of Learning-Based Speech Enhancement Systems in Noisy and Reverberant Environments
• FullSubNet: A Full-Band and Sub-Band Fusion Model for Real-Time Single-Channel Speech Enhancement
• Real-time Denoising and Dereverberation with Tiny Recurrent U-Net
• Transformers with Competitive Ensembles of Independent Mechanisms
• On Batching Variable Size Inputs for Training End-to-End Speech Enhancement Systems
• Training Speech Enhancement Systems with Noisy Speech Datasets
• Speech Enhancement Using Multi-Stage Self-Attentive Temporal Convolutional Networks
• Dynamic Attention Based Generative Adversarial Network with Phase Post-Processing for Speech Enhancement
• Inter-channel Conv-TasNet for multichannel speech enhancement
• Speech Enhancement using Separable Polling Attention and Global Layer Normalization followed with PReLU
• Cross-domain Single-channel Speech Enhancement Model with Bi-projection Fusion Module for Noise-robust ASR