Amortized Bethe Free Energy Minimization for Learning MRFs
arXiv:1906.06399
Abstract
We propose to learn deep undirected graphical models (i.e., MRFs) with a non-ELBO objective for which we can calculate exact gradients. In particular, we optimize a saddle-point objective deriving from the Bethe free energy approximation to the partition function. Unlike much recent work in approximate inference, the derived objective requires no sampling, and can be efficiently computed even for very expressive MRFs. We furthermore amortize this optimization with trained inference networks. Experimentally, we find that the proposed approach compares favorably with loopy belief propagation, but is faster, and it allows for attaining better held out log likelihood than other recent approximate inference schemes.
arXiv commentNeurIPS 2019
References in corpus (19)
- Adam: A Method for Stochastic Optimization
- Auto-Encoding Variational Bayes
- Semi-Supervised Classification with Graph Convolutional Networks
- Categorical Reparameterization with Gumbel-Softmax
- Pixel Recurrent Neural Networks
- Loopy Belief Propagation for Approximate Inference: An Empirical Study
- Variational Inference with Normalizing Flows
- The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
- Discriminative Embeddings of Latent Variable Models for Structured Data
- Gated Graph Sequence Neural Networks
- Neural Variational Inference and Learning in Belief Networks
- REBAR: Low-variance, unbiased gradient estimates for discrete latent variable models
- Variational inference for Monte Carlo objectives
- Belief Optimization for Binary Networks: A Stable Alternative to Loopy Belief Propagation
- Learning Approximate Inference Networks for Structured Prediction
- Convexifying the Bethe Free Energy
- A Tutorial on Deep Latent Variable Models of Natural Language
- Deeply Learning the Messages in Message Passing Inference
- Mean-Field Networks