DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models

Abstract

Diffusion probabilistic models (DPMs) have achieved impressive success in high-resolution image synthesis, especially in recent large-scale text-to-image generation applications. An essential technique for improving the sample quality of DPMs is guided sampling, which usually needs a large guidance scale to obtain the best sample quality. The commonly-used fast sampler for guided sampling is DDIM, a first-order diffusion ODE solver that generally needs 100 to 250 steps for high-quality samples. Although recent works propose dedicated high-order solvers and achieve a further speedup for sampling without guidance, their effectiveness for guided sampling has not been well-tested before. In this work, we demonstrate that previous high-order fast samplers suffer from instability issues, and they even become slower than DDIM when the guidance scale grows large. To further speed up guided sampling, we propose DPM-Solver++, a high-order solver for the guided sampling of DPMs. DPM-Solver++ solves the diffusion ODE with the data prediction model and adopts thresholding methods to keep the solution matches training data distribution. We further propose a multistep variant of DPM-Solver++ to address the instability issue by reducing the effective step size. Experiments show that DPM-Solver++ can generate high-quality samples within only 15 to 20 steps for guided sampling by pixel-space and latent-space DPMs.

Machine Intelligence Research

References in corpus (41)

• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Hierarchical Text-Conditional Image Generation with CLIP Latents
• Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding
• Score-Based Generative Modeling through Stochastic Differential Equations
• GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models
• Cascaded Diffusion Models for High Fidelity Image Generation
• Improved Denoising Diffusion Probabilistic Models
• Elucidating the Design Space of Diffusion-Based Generative Models
• SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations
• DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps
• Progressive Distillation for Fast Sampling of Diffusion Models
• GeoDiff: a Geometric Diffusion Model for Molecular Conformation Generation
• ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation
• Tackling the Generative Learning Trilemma with Denoising Diffusion GANs
• Pseudo Numerical Methods for Diffusion Models on Manifolds
• Equivariant Diffusion for Molecule Generation in 3D
• Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Models
• Knowledge Distillation in Iterative Generative Models for Improved Sampling Speed
• Fast Sampling of Diffusion Models with Exponential Integrator
• On Fast Sampling of Diffusion Probabilistic Models
• WaveGrad: Estimating Gradients for Waveform Generation
• Noise Estimation for Generative Diffusion Models
• UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Models
• Learning Fast Samplers for Diffusion Models by Differentiating Through Sample Quality
• Lossy Compression with Gaussian Diffusion
• Diffusion-based Molecule Generation with Informative Prior Bridges
• Estimating the Optimal Covariance with Imperfect Mean in Diffusion Probabilistic Models
• Fast Sampling of Diffusion Models via Operator Learning
• TRACT: Denoising Diffusion Models with Transitive Closure Time-Distillation
• Bilateral Denoising Diffusion Models
• DPM-Solver-v3: Improved Diffusion ODE Solver with Empirical Model Statistics
• Score-Based Generative Modeling with Critically-Damped Langevin Diffusion
• WaveGrad 2: Iterative Refinement for Text-to-Speech Synthesis
• Accelerating Guided Diffusion Sampling with Splitting Numerical Methods
• Multistep Consistency Models
• WorldSimBench: Towards Video Generation Models as World Simulators
• ERA-Solver: Error-Robust Adams Solver for Fast Sampling of Diffusion Probabilistic Models
• Vidu: a Highly Consistent, Dynamic and Skilled Text-to-Video Generator with Diffusion Models
• Simplifying, Stabilizing and Scaling Continuous-Time Consistency Models
• One-Step Diffusion Distillation through Score Implicit Matching
• A Unified Sampling Framework for Solver Searching of Diffusion Probabilistic Models

Cited by in corpus (5)

• FengWu: Pushing the Skillful Global Medium-range Weather Forecast beyond 10 Days Lead
• A Survey on Personalized Content Synthesis with Diffusion Models
• Break-the-Beat! Controllable MIDI-to-Drum Audio Synthesis
• Diffusion-based Galaxy Simulations for the Roman High Latitude Survey
• Stream-DiffVSR: Low-Latency Streamable Video Super-Resolution via Auto-Regressive Diffusion