Conjugate Generalized Bayesian Inference for Discrete Doubly Intractable Problems

Abstract

Doubly intractable problems occur when both the likelihood and the posterior are available only in unnormalized form, with computationally intractable normalization constants. Bayesian inference then typically requires direct approximation of the posterior through specialized and typically expensive MCMC methods. In this paper, we provide a computationally efficient alternative in the form of a novel generalized Bayesian posterior that allows for conjugate, closed-form or Gibbs-based MCMC inference within the class of exponential family models for discrete data. We derive theoretical guarantees to characterize the asymptotic behavior of the generalized posterior, supporting its use for inference. The method is evaluated on a range of challenging intractable exponential family models, including the Conway-Maxwell-Poisson graphical model of multivariate count data, autoregressive discrete time series models, and Markov random fields such as the Ising and Potts models. The computational gains are significant; in our experiments, the method is between 10 and 6000 times faster than state-of-the-art Bayesian computational methods.