Gotta Go Fast When Generating Data with Score-Based Models

Abstract

Score-based (denoising diffusion) generative models have recently gained a lot of success in generating realistic and diverse data. These approaches define a forward diffusion process for transforming data to noise and generate data by reversing it (thereby going from noise to data). Unfortunately, current score-based models generate data very slowly due to the sheer number of score network evaluations required by numerical SDE solvers. In this work, we aim to accelerate this process by devising a more efficient SDE solver. Existing approaches rely on the Euler-Maruyama (EM) solver, which uses a fixed step size. We found that naively replacing it with other SDE solvers fares poorly - they either result in low-quality samples or become slower than EM. To get around this issue, we carefully devise an SDE solver with adaptive step sizes tailored to score-based generative models piece by piece. Our solver requires only two score function evaluations, rarely rejects samples, and leads to high-quality samples. Our approach generates data 2 to 10 times faster than EM while achieving better or equal sample quality. For high-resolution images, our method leads to significantly higher quality samples than all other methods tested. Our SDE solver has the benefit of requiring no step size tuning.

Code is available on https://github.com/AlexiaJM/score_sde_fast_sampling

References in corpus (14)

• Denoising Diffusion Probabilistic Models
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
• Improved Denoising Diffusion Probabilistic Models
• Improved Techniques for Training Score-Based Generative Models
• DiffWave: A Versatile Diffusion Model for Audio Synthesis
• Denoising Diffusion Implicit Models
• UNIT-DDPM: UNpaired Image Translation with Denoising Diffusion Probabilistic Models
• Noise Estimation for Generative Diffusion Models
• Adversarial score matching and improved sampling for image generation
• Modify the Improved Euler scheme to integrate stochastic differential equations
• Solving Linear Inverse Problems Using the Prior Implicit in a Denoiser
• VoiceGrad: Non-Parallel Any-to-Many Voice Conversion with Annealed Langevin Dynamics
• Symbolic Music Generation with Diffusion Models

Cited by in corpus (6)

• Diffusion Models in Vision: A Survey
• Diffusion Schrödinger Bridge with Applications to Score-Based Generative Modeling
• How Much is Enough? A Study on Diffusion Times in Score-based Generative Models
• Likelihood Training of Schrödinger Bridge using Forward-Backward SDEs Theory
• Autoregressive Diffusion Models
• Cascading Modular Network (CAM-Net) for Multimodal Image Synthesis