Oops I Took A Gradient: Scalable Sampling for Discrete Distributions

Abstract

We propose a general and scalable approximate sampling strategy for probabilistic models with discrete variables. Our approach uses gradients of the likelihood function with respect to its discrete inputs to propose updates in a Metropolis-Hastings sampler. We show empirically that this approach outperforms generic samplers in a number of difficult settings including Ising models, Potts models, restricted Boltzmann machines, and factorial hidden Markov models. We also demonstrate the use of our improved sampler for training deep energy-based models on high dimensional discrete data. This approach outperforms variational auto-encoders and existing energy-based models. Finally, we give bounds showing that our approach is near-optimal in the class of samplers which propose local updates.

arXiv commentEnergy-Based Models, Deep generative models, MCMC sampling

References in corpus (13)

• Adam: A Method for Stochastic Optimization
• Generative Modeling by Estimating Gradients of the Data Distribution
• The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
• Searching for Activation Functions
• Implicit Generation and Generalization in Energy-Based Models
• Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One
• Discontinuous Hamiltonian Monte Carlo for discrete parameters and discontinuous likelihoods
• Learning the Stein Discrepancy for Training and Evaluating Energy-Based Models without Sampling
• Stein Variational Gradient Descent Without Gradient
• Stochastic Security: Adversarial Defense Using Long-Run Dynamics of Energy-Based Models
• No MCMC for me: Amortized sampling for fast and stable training of energy-based models
• Stein Variational Inference for Discrete Distributions
• Joint Energy-based Model Training for Better Calibrated Natural Language Understanding Models

Cited by in corpus (4)

• Flow Network based Generative Models for Non-Iterative Diverse Candidate Generation
• Exposing the Implicit Energy Networks behind Masked Language Models via Metropolis--Hastings
• The Taxicab Sampler: MCMC for Discrete Spaces with Application to Tree Models
• Perturb-and-max-product: Sampling and learning in discrete energy-based models