Non Gaussian Denoising Diffusion Models
arXiv:2106.07582
Abstract
Generative diffusion processes are an emerging and effective tool for image and speech generation. In the existing methods, the underline noise distribution of the diffusion process is Gaussian noise. However, fitting distributions with more degrees of freedom, could help the performance of such generative models. In this work, we investigate other types of noise distribution for the diffusion process. Specifically, we show that noise from Gamma distribution provides improved results for image and speech generation. Moreover, we show that using a mixture of Gaussian noise variables in the diffusion process improves the performance over a diffusion process that is based on a single distribution. Our approach preserves the ability to efficiently sample state in the training diffusion process while using Gamma noise and a mixture of noise.
References in corpus (12)
- Denoising Diffusion Probabilistic Models
- GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
- LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
- Score-Based Generative Modeling through Stochastic Differential Equations
- Generative Modeling by Estimating Gradients of the Data Distribution
- NVAE: A Deep Hierarchical Variational Autoencoder
- DiffWave: A Versatile Diffusion Model for Audio Synthesis
- Generating Diverse High-Fidelity Images with VQ-VAE-2
- High Fidelity Speech Synthesis with Adversarial Networks
- Denoising Diffusion Implicit Models
- WaveGrad: Estimating Gradients for Waveform Generation
- Learning Energy-Based Models by Diffusion Recovery Likelihood