On Sampling from the Gibbs Distribution with Random Maximum A-Posteriori Perturbations

Abstract

In this paper we describe how MAP inference can be used to sample efficiently from Gibbs distributions. Specifically, we provide means for drawing either approximate or unbiased samples from Gibbs' distributions by introducing low dimensional perturbations and solving the corresponding MAP assignments. Our approach also leads to new ways to derive lower bounds on partition functions. We demonstrate empirically that our method excels in the typical "high signal - high coupling" regime. The setting results in ragged energy landscapes that are challenging for alternative approaches to sampling and/or lower bounds.

References in corpus (4)

• Tightening LP Relaxations for MAP using Message Passing
• The Bethe Partition Function of Log-supermodular Graphical Models
• On the Partition Function and Random Maximum A-Posteriori Perturbations
• Optimization of Structured Mean Field Objectives

Cited by in corpus (8)

• Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety
• Gradient Estimation with Stochastic Softmax Tricks
• On Measure Concentration of Random Maximum A-Posteriori Perturbations
• Exact and approximate inference in graphical models: variable elimination and beyond
• Direct Optimization through $\arg \max$ for Discrete Variational Auto-Encoder
• Learning Maximum-A-Posteriori Perturbation Models for Structured Prediction in Polynomial Time
• Training Restricted Boltzmann Machine by Perturbation
• Unbiased Gradient Estimation with Balanced Assignments for Mixtures of Experts