Diffiner: A Versatile Diffusion-based Generative Refiner for Speech Enhancement
arXiv:2210.17287 · doi:10.21437/Interspeech.2023-1547
Abstract
Although deep neural network (DNN)-based speech enhancement (SE) methods outperform the previous non-DNN-based ones, they often degrade the perceptual quality of generated outputs. To tackle this problem, we introduce a DNN-based generative refiner, Diffiner, aiming to improve perceptual speech quality pre-processed by an SE method. We train a diffusion-based generative model by utilizing a dataset consisting of clean speech only. Then, our refiner effectively mixes clean parts newly generated via denoising diffusion restoration into the degraded and distorted parts caused by a preceding SE method, resulting in refined speech. Once our refiner is trained on a set of clean speech, it can be applied to various SE methods without additional training specialized for each SE module. Therefore, our refiner can be a versatile post-processing module w.r.t. SE methods and has high potential in terms of modularity. Experimental results show that our method improved perceptual speech quality regardless of the preceding SE methods used.
arXiv commentAccepted by Interspeech 2023
References in corpus (14)
- Adam: A Method for Stochastic Optimization
- Auto-Encoding Variational Bayes
- Denoising Diffusion Probabilistic Models
- Diffusion Models Beat GANs on Image Synthesis
- Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separation
- Variational Inference with Normalizing Flows
- Speech Enhancement and Dereverberation with Diffusion-based Generative Models
- Denoising Diffusion Restoration Models
- DNN-based Source Enhancement to Increase Objective Sound Quality Assessment Score
- Universal Speech Enhancement with Score-based Diffusion
- A Study on Speech Enhancement Based on Diffusion Probabilistic Model
- Restoring degraded speech via a modified diffusion model
- Discretization and Re-synthesis: an alternative method to solve the Cocktail Party Problem
- Real-time Denoising and Dereverberation with Tiny Recurrent U-Net