Neural Variational Inference and Learning in Undirected Graphical Models

Abstract

Many problems in machine learning are naturally expressed in the language of undirected graphical models. Here, we propose black-box learning and inference algorithms for undirected models that optimize a variational approximation to the log-likelihood of the model. Central to our approach is an upper bound on the log-partition function parametrized by a function q that we express as a flexible neural network. Our bound makes it possible to track the partition function during learning, to speed-up sampling, and to train a broad class of hybrid directed/undirected models via a unified variational inference framework. We empirically demonstrate the effectiveness of our method on several popular generative modeling datasets.

arXiv commentAppearing in Proceedings of the 31st Conference on Neural Information Processing Systems (NIPS) 2017, Long Beach, CA, USA

References in corpus (8)

• Adam: A Method for Stochastic Optimization
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• Theano: new features and speed improvements
• Markov Chain Monte Carlo and Variational Inference: Bridging the Gap
• Auxiliary Deep Generative Models
• Edward: A library for probabilistic modeling, inference, and criticism
• Variational inference for Monte Carlo objectives
• Discrete Variational Autoencoders