WaveGrad: Estimating Gradients for Waveform Generation

Abstract

This paper introduces WaveGrad, a conditional model for waveform generation which estimates gradients of the data density. The model is built on prior work on score matching and diffusion probabilistic models. It starts from a Gaussian white noise signal and iteratively refines the signal via a gradient-based sampler conditioned on the mel-spectrogram. WaveGrad offers a natural way to trade inference speed for sample quality by adjusting the number of refinement steps, and bridges the gap between non-autoregressive and autoregressive models in terms of audio quality. We find that it can generate high fidelity audio samples using as few as six iterations. Experiments reveal WaveGrad to generate high fidelity audio, outperforming adversarial non-autoregressive baselines and matching a strong likelihood-based autoregressive baseline using fewer sequential operations. Audio samples are available at https://wavegrad.github.io/.

References in corpus (21)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Denoising Diffusion Probabilistic Models
• Generative Modeling by Estimating Gradients of the Data Distribution
• MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis
• SampleRNN: An Unconditional End-to-End Neural Audio Generation Model
• Semantic Image Synthesis with Spatially-Adaptive Normalization
• GANSynth: Adversarial Neural Audio Synthesis
• Insertion Transformer: Flexible Sequence Generation via Insertion Operations
• Improved Techniques for Training Score-Based Generative Models
• DiffWave: A Versatile Diffusion Model for Audio Synthesis
• MelNet: A Generative Model for Audio in the Frequency Domain
• DDSP: Differentiable Digital Signal Processing
• Sliced Score Matching: A Scalable Approach to Density and Score Estimation
• KERMIT: Generative Insertion-Based Modeling for Sequences
• Deep Energy Estimator Networks
• Non-Autoregressive Neural Text-to-Speech
• A Spectral Energy Distance for Parallel Speech Synthesis
• HooliGAN: Robust, High Quality Neural Vocoding
• WaveNODE: A Continuous Normalizing Flow for Speech Synthesis
• Insertion-Deletion Transformer
• Knowledge-and-Data-Driven Amplitude Spectrum Prediction for Hierarchical Neural Vocoders

Cited by in corpus (5)

• Diffusion Models Beat GANs on Image Synthesis
• DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models
• Non Gaussian Denoising Diffusion Models
• Score-based Generative Modeling in Latent Space
• Diffusion models for Handwriting Generation