Adversarial score matching and improved sampling for image generation

Abstract

Denoising Score Matching with Annealed Langevin Sampling (DSM-ALS) has recently found success in generative modeling. The approach works by first training a neural network to estimate the score of a distribution, and then using Langevin dynamics to sample from the data distribution assumed by the score network. Despite the convincing visual quality of samples, this method appears to perform worse than Generative Adversarial Networks (GANs) under the Fréchet Inception Distance, a standard metric for generative models. We show that this apparent gap vanishes when denoising the final Langevin samples using the score network. In addition, we propose two improvements to DSM-ALS: 1) Consistent Annealed Sampling as a more stable alternative to Annealed Langevin Sampling, and 2) a hybrid training formulation, composed of both Denoising Score Matching and adversarial objectives. By combining these two techniques and exploring different network architectures, we elevate score matching methods and obtain results competitive with state-of-the-art image generation on CIFAR-10.

arXiv commentCode at https://github.com/AlexiaJM/AdversarialConsistentScoreMatching

References in corpus (22)

• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Denoising Diffusion Probabilistic Models
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Spectral Normalization for Generative Adversarial Networks
• Self-Attention Generative Adversarial Networks
• Deep Unsupervised Learning using Nonequilibrium Thermodynamics
• LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
• Generative Modeling by Estimating Gradients of the Data Distribution
• Adversarially Learned Inference
• ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness
• The relativistic discriminator: a key element missing from standard GAN
• VEEGAN: Reducing Mode Collapse in GANs using Implicit Variational Learning
• Unrolled Generative Adversarial Networks
• On Convergence and Stability of GANs
• Improved Techniques for Training Score-Based Generative Models
• HYPE: A Benchmark for Human eYe Perceptual Evaluation of Generative Models
• Maximum Entropy Generators for Energy-Based Models
• Prescribed Generative Adversarial Networks
• Solving Linear Inverse Problems Using the Prior Implicit in a Denoiser
• On Relativistic $f$-Divergences
• Blindness of score-based methods to isolated components and mixing proportions

Cited by in corpus (12)

• Diffusion Models Beat GANs on Image Synthesis
• Diffusion Models in Vision: A Survey
• Score-Based Generative Modeling through Stochastic Differential Equations
• Improved Denoising Diffusion Probabilistic Models
• Adversarial Text-to-Image Synthesis: A Review
• Improved Techniques for Training Score-Based Generative Models
• Denoising Diffusion Implicit Models
• NU-Wave: A Diffusion Probabilistic Model for Neural Audio Upsampling
• Gotta Go Fast When Generating Data with Score-Based Models
• Diffusion models for Handwriting Generation
• Diffusion Normalizing Flow
• Dual Training of Energy-Based Models with Overparametrized Shallow Neural Networks