A Variational Perspective on Diffusion-Based Generative Models and Score Matching

Abstract

Discrete-time diffusion-based generative models and score matching methods have shown promising results in modeling high-dimensional image data. Recently, Song et al. (2021) show that diffusion processes that transform data into noise can be reversed via learning the score function, i.e. the gradient of the log-density of the perturbed data. They propose to plug the learned score function into an inverse formula to define a generative diffusion process. Despite the empirical success, a theoretical underpinning of this procedure is still lacking. In this work, we approach the (continuous-time) generative diffusion directly and derive a variational framework for likelihood estimation, which includes continuous-time normalizing flows as a special case, and can be seen as an infinitely deep variational autoencoder. Under this framework, we show that minimizing the score-matching loss is equivalent to maximizing a lower bound of the likelihood of the plug-in reverse SDE proposed by Song et al. (2021), bridging the theoretical gap.

References in corpus (14)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Auto-Encoding Variational Bayes
• The NumPy array: a structure for efficient numerical computation
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Diffusion Models Beat GANs on Image Synthesis
• Variational Diffusion Models
• Improved Techniques for Training Score-Based Generative Models
• DiffWave: A Versatile Diffusion Model for Audio Synthesis
• How to Train Your Energy-Based Models
• Interpretation and Generalization of Score Matching
• Neural Stochastic Differential Equations: Deep Latent Gaussian Models in the Diffusion Limit
• Argmax Flows and Multinomial Diffusion: Learning Categorical Distributions
• A RAD approach to deep mixture models
• Infinitely Deep Bayesian Neural Networks with Stochastic Differential Equations

Cited by in corpus (12)

• Diffusion Models in Vision: A Survey
• Variational Diffusion Models
• Speech Enhancement and Dereverberation with Diffusion-based Generative Models
• StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Models
• How Much is Enough? A Study on Diffusion Times in Score-based Generative Models
• SegDiff: Image Segmentation with Diffusion Probabilistic Models
• Score-based Generative Modeling in Latent Space
• Choose your tools carefully: A Comparative Evaluation of Deterministic vs. Stochastic and Binary vs. Analog Neuron models for Implementing Emerging Computing Paradigms
• Likelihood Training of Schrödinger Bridge using Forward-Backward SDEs Theory
• Denoising Diffusion Gamma Models
• Simulating Diffusion Bridges with Score Matching
• Autoregressive Diffusion Models