FloWaveNet : A Generative Flow for Raw Audio

Abstract

Most modern text-to-speech architectures use a WaveNet vocoder for synthesizing high-fidelity waveform audio, but there have been limitations, such as high inference time, in its practical application due to its ancestral sampling scheme. The recently suggested Parallel WaveNet and ClariNet have achieved real-time audio synthesis capability by incorporating inverse autoregressive flow for parallel sampling. However, these approaches require a two-stage training pipeline with a well-trained teacher network and can only produce natural sound by using probability distillation along with auxiliary loss terms. We propose FloWaveNet, a flow-based generative model for raw audio synthesis. FloWaveNet requires only a single-stage training procedure and a single maximum likelihood loss, without any additional auxiliary terms, and it is inherently parallel due to the characteristics of generative flow. The model can efficiently sample raw audio in real-time, with clarity comparable to previous two-stage parallel models. The code and samples for all models, including our FloWaveNet, are publicly available.

9 pages, ICML'2019

References in corpus (9)

• Adam: A Method for Stochastic Optimization
• WaveNet: A Generative Model for Raw Audio
• Density estimation using Real NVP
• Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis
• Deep Voice: Real-time Neural Text-to-Speech
• Deep Voice 2: Multi-Speaker Neural Text-to-Speech
• Deep Voice 3: Scaling Text-to-Speech with Convolutional Sequence Learning
• ClariNet: Parallel Wave Generation in End-to-End Text-to-Speech
• Transformation Autoregressive Networks

Cited by in corpus (19)

• SoftFlow: Probabilistic Framework for Normalizing Flow on Manifolds
• IDF++: Analyzing and Improving Integer Discrete Flows for Lossless Compression
• Neural Spline Flows
• ConvS2S-VC: Fully convolutional sequence-to-sequence voice conversion
• A Spectral Energy Distance for Parallel Speech Synthesis
• Review of end-to-end speech synthesis technology based on deep learning
• Multi-Target Emotional Voice Conversion With Neural Vocoders
• VoiceGrad: Non-Parallel Any-to-Many Voice Conversion with Annealed Langevin Dynamics
• Many-to-Many Voice Transformer Network
• Neural Density Estimation and Likelihood-free Inference
• WaveNODE: A Continuous Normalizing Flow for Speech Synthesis
• Approaching the Limit of Image Rescaling via Flow Guidance
• A Survey on Audio Synthesis and Audio-Visual Multimodal Processing
• Problems using deep generative models for probabilistic audio source separation
• Nonparallel Voice Conversion with Augmented Classifier Star Generative Adversarial Networks
• Training a Neural Speech Waveform Model using Spectral Losses of Short-Time Fourier Transform and Continuous Wavelet Transform
• PeriodNet: A non-autoregressive waveform generation model with a structure separating periodic and aperiodic components
• Improving the expressiveness of neural vocoding with non-affine Normalizing Flows
• Empirical Evaluation of Deep Learning Model Compression Techniques on the WaveNet Vocoder