MCMC for doubly-intractable distributions

Abstract

Markov Chain Monte Carlo (MCMC) algorithms are routinely used to draw samples from distributions with intractable normalization constants. However, standard MCMC algorithms do not apply to doubly-intractable distributions in which there are additional parameter-dependent normalization terms; for example, the posterior over parameters of an undirected graphical model. An ingenious auxiliary-variable scheme (Moeller et al., 2004) offers a solution: exact sampling (Propp and Wilson, 1996) is used to sample from a Metropolis-Hastings proposal for which the acceptance probability is tractable. Unfortunately the acceptance probability of these expensive updates can be low. This paper provides a generalization of Moeller et al. (2004) and a new MCMC algorithm, which obtains better acceptance probabilities for the same amount of exact sampling, and removes the need to estimate model parameters before sampling begins.

Appears in Proceedings of the Twenty-Second Conference on Uncertainty in Artificial Intelligence (UAI2006)

References in corpus (4)

• Equilibrium free energy differences from nonequilibrium measurements: a master equation approach
• Taking Bigger Metropolis Steps by Dragging Fast Variables
• Estimating Ratios of Normalizing Constants Using Linked Importance Sampling
• Exact sampling from non-attractive distributions using summary states

Cited by in corpus (71)

• Approximate Bayesian computation (ABC) gives exact results under the assumption of model error
• BDgraph: An R Package for Bayesian Structure Learning in Graphical Models
• On Russian Roulette Estimates for Bayesian Inference with Doubly-Intractable Likelihoods
• Estimating the granularity coefficient of a Potts-Markov random field within an MCMC algorithm
• Inverse Reward Design
• Bayesian Gaussian Copula Graphical Modeling for Dupuytren Disease
• Adaptive Bayesian Radio Tomography
• Pre-processing for approximate Bayesian computation in image analysis
• Statistical Inference for Generative Models with Maximum Mean Discrepancy
• Coupled MCMC with a randomized acceptance probability
• Efficient Bayesian inference for exponential random graph models by correcting the pseudo-posterior distribution
• On the normalized power prior
• Perturbation Bounds for Monte Carlo within Metropolis via Restricted Approximations
• Metropolis Sampling
• Pseudo-Marginal Slice Sampling
• Classification using distance nearest neighbours
• Stochastic Approximation Cut Algorithm for Inference in Modularized Bayesian Models
• Probabilistic preference learning with the Mallows rank model
• Bayesian exponential family projections for coupled data sources
• Bayesian object classification of gold nanoparticles
• Bayesian computation: a perspective on the current state, and sampling backwards and forwards
• The G-Wishart Weighted Proposal Algorithm: Efficient Posterior Computation for Gaussian Graphical Models
• Unbiased approximation of posteriors via coupled particle Markov chain Monte Carlo
• Bootstrapped synthetic likelihood
• Parsimonious Hierarchical Modeling Using Repulsive Distributions
• Nonparametric Bayesian Density Modeling with Gaussian Processes
• An external field prior for the hidden Potts model, with application to cone-beam computed tomography
• Marginal sequential Monte Carlo for doubly intractable models
• Bayesian Structure Learning for Markov Random Fields with a Spike and Slab Prior
• Efficient parameter sampling for Markov jump processes
• Model comparison for Gibbs random fields using noisy reversible jump Markov chain Monte Carlo
• The Nonlinearity Coefficient - A Practical Guide to Neural Architecture Design
• Metropolis-Hastings with Averaged Acceptance Ratios
• Fast model-fitting of Bayesian variable selection regression using the iterative complex factorization algorithm
• Nesting Probabilistic Programs
• Joint Segmentation and Deconvolution of Ultrasound Images Using a Hierarchical Bayesian Model based on Generalized Gaussian Priors
• Efficient Bayesian Inference of Sigmoidal Gaussian Cox Processes
• Light and Widely Applicable MCMC: Approximate Bayesian Inference for Large Datasets
• Bayesian Structure Learning for Markov Random Fields with a Spike and Slab Prior
• Gaussian Process-Based Bayesian Nonparametric Inference of Population Trajectories from Gene Genealogies
• Bayesian inference for exponential random graph models
• Stratified sampling and bootstrapping for approximate Bayesian computation
• A Bayesian Mark Interaction Model for Analysis of Tumor Pathology Images
• A Kernel Approach to Tractable Bayesian Nonparametrics
• Change point detection in dynamic Gaussian graphical models: the impact of COVID-19 pandemic on the US stock market
• Dynamic Models of Animal Movement with Spatial Point Process Interactions
• Kernel-based Approximate Bayesian Inference for Exponential Family Random Graph Models
• Computing the Cramer-Rao bound of Markov random field parameters: Application to the Ising and the Potts models
• On a wider class of prior distributions for graphical models
• Bayesian Hierarchical Spatial Model for Small Area Estimation with Non-ignorable Nonresponses and Its Applications to the NHANES Dental Caries Assessments
• Exact Bayesian Inference for the Bingham Distribution
• Bayesian Analysis of High-dimensional Discrete Graphical Models
• Efficient sampling of Gaussian graphical models using conditional Bayes factors
• Warped Gradient-Enhanced Gaussian Process Surrogate Models for Exponential Family Likelihoods with Intractable Normalizing Constants
• Evidence and Bayes factor estimation for Gibbs random fields
• MLE-induced Likelihood for Markov Random Fields
• Estimation and uncertainty quantification for the output from quantum simulators
• Econometric Models of Network Formation
• Bayesian model selection approach for colored graphical Gaussian models
• An ABC interpretation of the multiple auxiliary variable method
• Stable Exponential Random Graph Models with Non-parametric Components for Large Dense Networks
• A mixture model approach to infer land-use influence on point referenced water quality
• An Approach to Causal Inference over Stochastic Networks
• A joint model for authors characteristics and collaboration pattern in bibliometric networks: a Bayesian approach
• Bayesian density regression for count data
• Exploiting Multi-Core Architectures for Reduced-Variance Estimation with Intractable Likelihoods
• Measuring Sample Quality in Algorithms for Intractable Normalizing Function Problems
• Efficient Estimation of the number of neighbours in Probabilistic K Nearest Neighbour Classification
• A Multivariate Graphical Stochastic Volatility Model
• Flexible Mixture Modeling on Constrained Spaces
• Bayesian Multi-scale Modeling of Factor Matrix without using Partition Tree