Generative Modeling with Denoising Auto-Encoders and Langevin Sampling
arXiv:2002.00107
Abstract
We study convergence of a generative modeling method that first estimates the score function of the distribution using Denoising Auto-Encoders (DAE) or Denoising Score Matching (DSM) and then employs Langevin diffusion for sampling. We show that both DAE and DSM provide estimates of the score of the Gaussian smoothed population density, allowing us to apply the machinery of Empirical Processes. We overcome the challenge of relying only on bounds on the score estimation error and provide finite-sample bounds in the Wasserstein distance between the law of the population distribution and the law of this sampling scheme. We then apply our results to the homotopy method of arXiv:1907.05600 and provide theoretical justification for its empirical success.
References in corpus (5)
- Generative Modeling by Estimating Gradients of the Data Distribution
- Transportation cost-information inequalities and applications to random dynamical systems and diffusions
- Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One
- Norm-Based Capacity Control in Neural Networks
- Stochastic Runge-Kutta Accelerates Langevin Monte Carlo and Beyond
Cited by in corpus (7)
- Diffusion Schrödinger Bridge with Applications to Score-Based Generative Modeling
- Fast Mixing of Multi-Scale Langevin Dynamics under the Manifold Hypothesis
- On Energy-Based Models with Overparametrized Shallow Neural Networks
- Joint Intensity-Gradient Guided Generative Modeling for Colorization
- Inference and De-Noising of Non-Gaussian Particle Distribution Functions: A Generative Modeling Approach
- High-dimensional Assisted Generative Model for Color Image Restoration
- Homotopic Gradients of Generative Density Priors for MR Image Reconstruction