Understanding Diffusion Models: A Unified Perspective

Abstract

Diffusion models have shown incredible capabilities as generative models; indeed, they power the current state-of-the-art models on text-conditioned image generation such as Imagen and DALL-E 2. In this work we review, demystify, and unify the understanding of diffusion models across both variational and score-based perspectives. We first derive Variational Diffusion Models (VDM) as a special case of a Markovian Hierarchical Variational Autoencoder, where three key assumptions enable tractable computation and scalable optimization of the ELBO. We then prove that optimizing a VDM boils down to learning a neural network to predict one of three potential objectives: the original source input from any arbitrary noisification of it, the original source noise from any arbitrarily noisified input, or the score function of a noisified input at any arbitrary noise level. We then dive deeper into what it means to learn the score function, and connect the variational perspective of a diffusion model explicitly with the Score-based Generative Modeling perspective through Tweedie's Formula. Lastly, we cover how to learn a conditional distribution using diffusion models via guidance.

Cited by in corpus (10)

• Diffusion Models, Image Super-Resolution And Everything: A Survey
• Ensemble flow reconstruction in the atmospheric boundary layer from spatially limited measurements through latent diffusion models
• Machine-Learning-Assisted Photonic Device Development: A Multiscale Approach from Theory to Characterization
• Balancing User Preferences by Social Networks: A Condition-Guided Social Recommendation Model for Mitigating Popularity Bias
• GenSelfDiff-HIS: Generative Self-Supervision Using Diffusion for Histopathological Image Segmentation
• Physics-informed generative model for drug-like molecule conformers
• Generative Design of Crystal Structures by Point Cloud Representations and Diffusion Model
• Multi-Resolution Diffusion for Privacy-Sensitive Recommender Systems
• Self-Supervised Adversarial Diffusion Models for Fast MRI Reconstruction
• Back-Projection Diffusion: Solving the Wideband Inverse Scattering Problem with Diffusion Models