NU-GAN: High resolution neural upsampling with GAN
arXiv:2010.11362
Abstract
In this paper, we propose NU-GAN, a new method for resampling audio from lower to higher sampling rates (upsampling). Audio upsampling is an important problem since productionizing generative speech technology requires operating at high sampling rates. Such applications use audio at a resolution of 44.1 kHz or 48 kHz, whereas current speech synthesis methods are equipped to handle a maximum of 24 kHz resolution. NU-GAN takes a leap towards solving audio upsampling as a separate component in the text-to-speech (TTS) pipeline by leveraging techniques for audio generation using GANs. ABX preference tests indicate that our NU-GAN resampler is capable of resampling 22 kHz to 44.1 kHz audio that is distinguishable from original audio only 7.4% higher than random chance for single speaker dataset, and 10.8% higher than chance for multi-speaker dataset.
References in corpus (8)
- WaveNet: A Generative Model for Raw Audio
- Spectral Normalization for Generative Adversarial Networks
- MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis
- SampleRNN: An Unconditional End-to-End Neural Audio Generation Model
- GANSynth: Adversarial Neural Audio Synthesis
- Tacotron: Towards End-to-End Speech Synthesis
- ClariNet: Parallel Wave Generation in End-to-End Text-to-Speech
- Audio Super Resolution using Neural Networks
Cited by in corpus (5)
- VoiceFixer: A Unified Framework for High-Fidelity Speech Restoration
- Neural Vocoder is All You Need for Speech Super-resolution
- Stochastic Restoration of Heavily Compressed Musical Audio using Generative Adversarial Networks
- VoiceFixer: Toward General Speech Restoration with Neural Vocoder
- TTS-Portuguese Corpus: a corpus for speech synthesis in Brazilian Portuguese