Variational Inference In Pachinko Allocation Machines
arXiv:1804.07944
Abstract
The Pachinko Allocation Machine (PAM) is a deep topic model that allows representing rich correlation structures among topics by a directed acyclic graph over topics. Because of the flexibility of the model, however, approximate inference is very difficult. Perhaps for this reason, only a small number of potential PAM architectures have been explored in the literature. In this paper we present an efficient and flexible amortized variational inference method for PAM, using a deep inference network to parameterize the approximate posterior distribution in a manner similar to the variational autoencoder. Our inference method produces more coherent topics than state-of-art inference methods for PAM while being an order of magnitude faster, which allows exploration of a wider range of PAM architectures than have previously been studied.
References in corpus (7)
- Adam: A Method for Stochastic Optimization
- Scikit-learn: Machine Learning in Python
- Neural Discrete Representation Learning
- Stochastic Backpropagation and Approximate Inference in Deep Generative Models
- Automatic Differentiation Variational Inference
- Conjugate-Computation Variational Inference : Converting Variational Inference in Non-Conjugate Models to Inferences in Conjugate Models
- The Generalized Reparameterization Gradient