Quasi-Periodic WaveNet: An Autoregressive Raw Waveform Generative Model with Pitch-dependent Dilated Convolution Neural Network

Abstract

In this paper, a pitch-adaptive waveform generative model named Quasi-Periodic WaveNet (QPNet) is proposed to improve the limited pitch controllability of vanilla WaveNet (WN) using pitch-dependent dilated convolution neural networks (PDCNNs). Specifically, as a probabilistic autoregressive generation model with stacked dilated convolution layers, WN achieves high-fidelity audio waveform generation. However, the pure-data-driven nature and the lack of prior knowledge of audio signals degrade the pitch controllability of WN. For instance, it is difficult for WN to precisely generate the periodic components of audio signals when the given auxiliary fundamental frequency ($F_0$) features are outside the $F_0$ range observed in the training data. To address this problem, QPNet with two novel designs is proposed. First, the PDCNN component is applied to dynamically change the network architecture of WN according to the given auxiliary $F_0$ features. Second, a cascaded network structure is utilized to simultaneously model the long- and short-term dependencies of quasi-periodic signals such as speech. The performances of single-tone sinusoid and speech generations are evaluated. The experimental results show the effectiveness of the PDCNNs for unseen auxiliary $F_0$ features and the effectiveness of the cascaded structure for speech generation.

arXiv comment15 pages, 12 figures, 11 tables

References in corpus (9)

• Adam: A Method for Stochastic Optimization
• WaveNet: A Generative Model for Raw Audio
• NICE: Non-linear Independent Components Estimation
• Variational Inference with Normalizing Flows
• SampleRNN: An Unconditional End-to-End Neural Audio Generation Model
• Parallel WaveNet: Fast High-Fidelity Speech Synthesis
• The challenge of realistic music generation: modelling raw audio at scale
• Quasi-Periodic WaveNet Vocoder: A Pitch Dependent Dilated Convolution Model for Parametric Speech Generation
• Non-parallel Voice Conversion System with WaveNet Vocoder and Collapsed Speech Suppression

Cited by in corpus (4)

• The Ethical Implications of Generative Audio Models: A Systematic Literature Review
• Quasi-Periodic Parallel WaveGAN Vocoder: A Non-autoregressive Pitch-dependent Dilated Convolution Model for Parametric Speech Generation
• Unified Source-Filter GAN: Unified Source-filter Network Based On Factorization of Quasi-Periodic Parallel WaveGAN
• A Cyclical Post-filtering Approach to Mismatch Refinement of Neural Vocoder for Text-to-speech Systems