Neural Vocoder is All You Need for Speech Super-resolution
arXiv:2203.14941 · doi:10.21437/Interspeech.2022-11017
Abstract
Speech super-resolution (SR) is a task to increase speech sampling rate by generating high-frequency components. Existing speech SR methods are trained in constrained experimental settings, such as a fixed upsampling ratio. These strong constraints can potentially lead to poor generalization ability in mismatched real-world cases. In this paper, we propose a neural vocoder based speech super-resolution method (NVSR) that can handle a variety of input resolution and upsampling ratios. NVSR consists of a mel-bandwidth extension module, a neural vocoder module, and a post-processing module. Our proposed system achieves state-of-the-art results on the VCTK multi-speaker benchmark. On 44.1 kHz target resolution, NVSR outperforms WSRGlow and Nu-wave by 8% and 37% respectively on log spectral distance and achieves a significantly better perceptual quality. We also demonstrate that prior knowledge in the pre-trained vocoder is crucial for speech SR by performing mel-bandwidth extension with a simple replication-padding method. Samples can be found in https://haoheliu.github.io/nvsr.
Submitted to INTERSPEECH 2022
References in corpus (8)
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis
- Decoupling Magnitude and Phase Estimation with Deep ResUNet for Music Source Separation
- VoiceFixer: Toward General Speech Restoration with Neural Vocoder
- NU-GAN: High resolution neural upsampling with GAN
- TFGAN: Time and Frequency Domain Based Generative Adversarial Network for High-fidelity Speech Synthesis
- CWS-PResUNet: Music Source Separation with Channel-wise Subband Phase-aware ResUNet
- Learning Continuous Representation of Audio for Arbitrary Scale Super Resolution