Region-based Energy Neural Network for Approximate Inference

Abstract

Region-based free energy was originally proposed for generalized belief propagation (GBP) to improve loopy belief propagation (loopy BP). In this paper, we propose a neural network based energy model for inference in general Markov random fields (MRFs), which directly minimizes the region-based free energy defined on region graphs. We term our model Region-based Energy Neural Network (RENN). Unlike message-passing algorithms, RENN avoids iterative message propagation and is faster. Also different from recent deep neural network based models, inference by RENN does not require sampling, and RENN works on general MRFs. RENN can also be employed for MRF learning. Our experiments on marginal distribution estimation, partition function estimation, and learning of MRFs show that RENN outperforms the mean field method, loopy BP, GBP, and the state-of-the-art neural network based model.

References in corpus (10)

• Auto-Encoding Variational Bayes
• Neural Message Passing for Quantum Chemistry
• Sufficient conditions for convergence of the Sum-Product Algorithm
• Composing graphical models with neural networks for structured representations and fast inference
• GMNN: Graph Markov Neural Networks
• Belief Optimization for Binary Networks: A Stable Alternative to Loopy Belief Propagation
• Structured Region Graphs: Morphing EP into GBP
• Graphical Generative Adversarial Networks
• Generalized Belief Propagation on Tree Robust Structured Region Graphs
• On the Choice of Regions for Generalized Belief Propagation