Real-time Denoising and Dereverberation with Tiny Recurrent U-Net

Abstract

Modern deep learning-based models have seen outstanding performance improvement with speech enhancement tasks. The number of parameters of state-of-the-art models, however, is often too large to be deployed on devices for real-world applications. To this end, we propose Tiny Recurrent U-Net (TRU-Net), a lightweight online inference model that matches the performance of current state-of-the-art models. The size of the quantized version of TRU-Net is 362 kilobytes, which is small enough to be deployed on edge devices. In addition, we combine the small-sized model with a new masking method called phase-aware $\beta$-sigmoid mask, which enables simultaneous denoising and dereverberation. Results of both objective and subjective evaluations have shown that our model can achieve competitive performance with the current state-of-the-art models on benchmark datasets using fewer parameters by orders of magnitude.

arXiv comment5 pages, 2 figures, 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). arXiv admin note: text overlap with arXiv:2006.00687

References in corpus (9)

• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Categorical Reparameterization with Gumbel-Softmax
• On the Convergence of Adam and Beyond
• Quantizing deep convolutional networks for efficient inference: A whitepaper
• Pyroomacoustics: A Python package for audio room simulations and array processing algorithms
• TinyLSTMs: Efficient Neural Speech Enhancement for Hearing Aids
• Phase-aware Speech Enhancement with Deep Complex U-Net
• DDSP: Differentiable Digital Signal Processing
• Exploring the Best Loss Function for DNN-Based Low-latency Speech Enhancement with Temporal Convolutional Networks

Cited by in corpus (4)

• Glance and Gaze: A Collaborative Learning Framework for Single-channel Speech Enhancement
• Diffiner: A Versatile Diffusion-based Generative Refiner for Speech Enhancement
• Training Speech Enhancement Systems with Noisy Speech Datasets
• A Deep Learning Loss Function based on Auditory Power Compression for Speech Enhancement