Improved Techniques for Training Score-Based Generative Models

Abstract

Score-based generative models can produce high quality image samples comparable to GANs, without requiring adversarial optimization. However, existing training procedures are limited to images of low resolution (typically below 32x32), and can be unstable under some settings. We provide a new theoretical analysis of learning and sampling from score models in high dimensional spaces, explaining existing failure modes and motivating new solutions that generalize across datasets. To enhance stability, we also propose to maintain an exponential moving average of model weights. With these improvements, we can effortlessly scale score-based generative models to images with unprecedented resolutions ranging from 64x64 to 256x256. Our score-based models can generate high-fidelity samples that rival best-in-class GANs on various image datasets, including CelebA, FFHQ, and multiple LSUN categories.

NeurIPS 2020

References in corpus (5)

• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• On the Convergence of Adam and Beyond
• Sliced Score Matching: A Scalable Approach to Density and Score Estimation
• Adversarial score matching and improved sampling for image generation
• Solving Linear Inverse Problems Using the Prior Implicit in a Denoiser

Cited by in corpus (39)

• Denoising Diffusion Probabilistic Models
• Diffusion Models Beat GANs on Image Synthesis
• Cascaded Diffusion Models for High Fidelity Image Generation
• Improved Denoising Diffusion Probabilistic Models
• DiffWave: A Versatile Diffusion Model for Audio Synthesis
• Denoising Diffusion Implicit Models
• Structured Denoising Diffusion Models in Discrete State-Spaces
• How to Train Your Energy-Based Models
• NU-Wave: A Diffusion Probabilistic Model for Neural Audio Upsampling
• Diffusion Schrödinger Bridge with Applications to Score-Based Generative Modeling
• On Fast Sampling of Diffusion Probabilistic Models
• Robust Compressed Sensing MRI with Deep Generative Priors
• WaveGrad: Estimating Gradients for Waveform Generation
• Autoregressive Denoising Diffusion Models for Multivariate Probabilistic Time Series Forecasting
• A Variational Perspective on Diffusion-Based Generative Models and Score Matching
• MatFusion: A Generative Diffusion Model for SVBRDF Capture
• Adversarial score matching and improved sampling for image generation
• Generative machine learning methods for multivariate ensemble post-processing
• Review of end-to-end speech synthesis technology based on deep learning
• Gotta Go Fast When Generating Data with Score-Based Models
• Learning Energy-Based Models by Diffusion Recovery Likelihood
• ScoreGrad: Multivariate Probabilistic Time Series Forecasting with Continuous Energy-based Generative Models
• Bilateral Denoising Diffusion Models
• VoiceGrad: Non-Parallel Any-to-Many Voice Conversion with Annealed Langevin Dynamics
• Symbolic Music Generation with Diffusion Models
• PROUD: PaRetO-gUided Diffusion Model for Multi-objective Generation
• No MCMC for me: Amortized sampling for fast and stable training of energy-based models
• Multiscale Score Matching for Out-of-Distribution Detection
• VAEBM: A Symbiosis between Variational Autoencoders and Energy-based Models
• Likelihood Training of Schrödinger Bridge using Forward-Backward SDEs Theory
• Quantum Algorithms for the Pathwise Lasso
• Improved Autoregressive Modeling with Distribution Smoothing
• Deep Generative Learning via Schrödinger Bridge
• Adaptive and Iterative Point Cloud Denoising with Score-Based Diffusion Model
• Density Ratio Estimation via Infinitesimal Classification
• Diffusion models for Handwriting Generation
• Autoencoding Under Normalization Constraints
• Probabilistic Mapping of Dark Matter by Neural Score Matching
• Diffusion Normalizing Flow