Parallel WaveNet: Fast High-Fidelity Speech Synthesis

Abstract

The recently-developed WaveNet architecture is the current state of the art in realistic speech synthesis, consistently rated as more natural sounding for many different languages than any previous system. However, because WaveNet relies on sequential generation of one audio sample at a time, it is poorly suited to today's massively parallel computers, and therefore hard to deploy in a real-time production setting. This paper introduces Probability Density Distillation, a new method for training a parallel feed-forward network from a trained WaveNet with no significant difference in quality. The resulting system is capable of generating high-fidelity speech samples at more than 20 times faster than real-time, and is deployed online by Google Assistant, including serving multiple English and Japanese voices.

Cited by in corpus (38)

• Knowledge Distillation: A Survey
• Normalizing Flows: An Introduction and Review of Current Methods
• Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data
• Deep Learning for Audio Signal Processing
• Flow-based generative models for Markov chain Monte Carlo in lattice field theory
• Neural Network Renormalization Group
• Emotion Intensity and its Control for Emotional Voice Conversion
• Fast Spectrogram Inversion using Multi-head Convolutional Neural Networks
• An Overview of Affective Speech Synthesis and Conversion in the Deep Learning Era
• End-to-End Video-To-Speech Synthesis using Generative Adversarial Networks
• Complex-valued neural networks for machine learning on non-stationary physical data
• Computer-assisted Pronunciation Training -- Speech synthesis is almost all you need
• The Effect of Explicit Structure Encoding of Deep Neural Networks for Symbolic Music Generation
• Short-Term Density Forecasting of Low-Voltage Load using Bernstein-Polynomial Normalizing Flows
• Quasi-Periodic WaveNet: An Autoregressive Raw Waveform Generative Model with Pitch-dependent Dilated Convolution Neural Network
• An Initial Investigation for Detecting Vocoder Fingerprints of Fake Audio
• Neural Canonical Transformation with Symplectic Flows
• High Quality Streaming Speech Synthesis with Low, Sentence-Length-Independent Latency
• WavThruVec: Latent speech representation as intermediate features for neural speech synthesis
• SQuId: Measuring Speech Naturalness in Many Languages
• Learning to Maximize Speech Quality Directly Using MOS Prediction for Neural Text-to-Speech
• Emotional Prosody Control for Speech Generation
• Autovocoder: Fast Waveform Generation from a Learned Speech Representation using Differentiable Digital Signal Processing
• Creating New Voices using Normalizing Flows
• ContextSpeech: Expressive and Efficient Text-to-Speech for Paragraph Reading
• Parallel Synthesis for Autoregressive Speech Generation
• Fitting Auditory Filterbanks with Multiresolution Neural Networks
• Spectrogram Inpainting for Interactive Generation of Instrument Sounds
• Improving Sequential Latent Variable Models with Autoregressive Flows
• Non-parallel Voice Conversion System with WaveNet Vocoder and Collapsed Speech Suppression
• An Exploratory Study on Perceptual Spaces of the Singing Voice
• LVCNet: Efficient Condition-Dependent Modeling Network for Waveform Generation
• Puffin: pitch-synchronous neural waveform generation for fullband speech on modest devices
• A Mathematical Framework for Learning Probability Distributions
• MelGlow: Efficient Waveform Generative Network Based on Location-Variable Convolution
• NeuralDPS: Neural Deterministic Plus Stochastic Model with Multiband Excitation for Noise-Controllable Waveform Generation
• Semi-supervised learning of glottal pulse positions in a neural analysis-synthesis framework
• Variational latent discrete representation for time series modelling