Correcting beliefs in the mean-field and Bethe approximations using linear response
arXiv:1302.1911 · doi:10.1109/ICCW.2013.6649462
Abstract
Approximating marginals of a graphical model is one of the fundamental problems in the theory of networks. In a recent paper a method was shown to construct a variational free energy such that the linear response estimates, and maximum entropy estimates (for beliefs) are in agreement, with implications for direct and inverse Ising problems[1]. In this paper we demonstrate an extension of that method, incorporating new information from the response matrix, and we recover the adaptive-TAP equations as the first order approximation[2]. The method is flexible with respect to applications of the cluster variational method, special cases of this method include Naive Mean Field (NMF) and Bethe. We demonstrate that the new framework improves estimation of marginals by orders of magnitude over standard implementations in the weak coupling limit. Beyond the weakly coupled regime we show there is an improvement in a model where the NMF and Bethe approximations are known to be poor for reasons of frustration and short loops.
arXiv comment5 pages, 1 figure, submitted to NETSTAT conference
References in corpus (8)
- Cluster Variation Method in Statistical Physics and Probabilistic Graphical Models
- Small-correlation expansions for the inverse Ising problem
- The Bethe approximation for solving the inverse Ising problem: a comparison with other inference methods
- Loop Calculus in Statistical Physics and Information Science
- How to Compute Loop Corrections to Bethe Approximation
- On the cavity method for decimated random constraint satisfaction problems and the analysis of belief propagation guided decimation algorithms
- Region graph partition function expansion and approximate free energy landscapes: Theory and some numerical results
- Susceptibility Propagation by Using Diagonal Consistency