Enhancing Low-Quality Voice Recordings Using Disentangled Channel Factor and Neural Waveform Model
arXiv:2011.05038
Abstract
High-quality speech corpora are essential foundations for most speech applications. However, such speech data are expensive and limited since they are collected in professional recording environments. In this work, we propose an encoder-decoder neural network to automatically enhance low-quality recordings to professional high-quality recordings. To address channel variability, we first filter out the channel characteristics from the original input audio using the encoder network with adversarial training. Next, we disentangle the channel factor from a reference audio. Conditioned on this factor, an auto-regressive decoder is then used to predict the target-environment Mel spectrogram. Finally, we apply a neural vocoder to synthesize the speech waveform. Experimental results show that the proposed system can generate a professional high-quality speech waveform when setting high-quality audio as the reference. It also improves speech enhancement performance compared with several state-of-the-art baseline systems.
arXiv comment8 pages. Accepted to IEEE SLT 2021
References in corpus (7)
- Adam: A Method for Stochastic Optimization
- Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis
- Efficient Neural Audio Synthesis
- HiFi-GAN: High-Fidelity Denoising and Dereverberation Based on Speech Deep Features in Adversarial Networks
- Noise Adaptive Speech Enhancement using Domain Adversarial Training
- Noise Tokens: Learning Neural Noise Templates for Environment-Aware Speech Enhancement
- Reverberation Modeling for Source-Filter-based Neural Vocoder