Explaining the effects of non-convergent sampling in the training of Energy-Based Models

Abstract

In this paper, we quantify the impact of using non-convergent Markov chains to train Energy-Based models (EBMs). In particular, we show analytically that EBMs trained with non-persistent short runs to estimate the gradient can perfectly reproduce a set of empirical statistics of the data, not at the level of the equilibrium measure, but through a precise dynamical process. Our results provide a first-principles explanation for the observations of recent works proposing the strategy of using short runs starting from random initial conditions as an efficient way to generate high-quality samples in EBMs, and lay the groundwork for using EBMs as diffusion models. After explaining this effect in generic EBMs, we analyze two solvable models in which the effect of the non-convergent sampling in the trained parameters can be described in detail. Finally, we test these predictions numerically on a ConvNet EBM and a Boltzmann machine.

arXiv commentAccepted at ICML 2023

References in corpus (22)

• Denoising Diffusion Probabilistic Models
• Solving the Quantum Many-Body Problem with Artificial Neural Networks
• Diffusion Models Beat GANs on Image Synthesis
• Direct-coupling analysis of residue co-evolution captures native contacts across many protein families
• Deep Unsupervised Learning using Nonequilibrium Thermodynamics
• Inverse statistical problems: from the inverse Ising problem to data science
• A Theory of Generative ConvNet
• The Bethe approximation for solving the inverse Ising problem: a comparison with other inference methods
• Deep Directed Generative Models with Energy-Based Probability Estimation
• Restricted Boltzmann Machine, recent advances and mean-field theory
• Maximum Entropy Generators for Energy-Based Models
• Thermodynamics of Restricted Boltzmann Machines and related learning dynamics
• Spectral Dynamics of Learning Restricted Boltzmann Machines
• Bethe-Peierls approximation and the inverse Ising model
• Equilibrium and non-Equilibrium regimes in the learning of Restricted Boltzmann Machines
• Learning a Restricted Boltzmann Machine using biased Monte Carlo sampling
• Structure and eigenvalues of heat-bath Markov chains
• Measuring complexity with zippers
• A Tale of Two Flows: Cooperative Learning of Langevin Flow and Normalizing Flow Toward Energy-Based Model
• Gaussian-Bernoulli RBMs Without Tears
• Robust Multi-Output Learning with Highly Incomplete Data via Restricted Boltzmann Machines
• MCMC Should Mix: Learning Energy-Based Model with Neural Transport Latent Space MCMC