Improving variational methods via pairwise linear response identities
arXiv:1611.00683
Abstract
Inference methods are often formulated as variational approximations: these approximations allow easy evaluation of statistics by marginalization or linear response, but these estimates can be inconsistent. We show that by introducing constraints on covariance, one can ensure consistency of linear response with the variational parameters, and in so doing inference of marginal probability distributions is improved. For the Bethe approximation and its generalizations, improvements are achieved with simple choices of the constraints. The approximations are presented as variational frameworks; iterative procedures related to message passing are provided for finding the minima.
arXiv comment36 pages, 17 figures
References in corpus (10)
- Tractable approximations for probabilistic models: The adaptive TAP mean field approach
- Approximate Inference and Constrained Optimization
- Solving Constraint Satisfaction Problems through Belief Propagation-guided decimation
- Loop Calculus in Statistical Physics and Information Science
- Linear Response Methods for Accurate Covariance Estimates from Mean Field Variational Bayes
- Susceptibility Propagation by Using Diagonal Consistency
- Characterizing and Improving Generalized Belief Propagation Algorithms on the 2D Edwards-Anderson Model
- A mean field method with correlations determined by linear response
- Adaptive Thouless-Anderson-Palmer approach to inverse Ising problems with quenched random fields
- Perturbative Corrections for Approximate Inference in Gaussian Latent Variable Models