Inferring Parameters and Structure of Latent Variable Models by Variational Bayes

Abstract

Current methods for learning graphical models with latent variables and a fixed structure estimate optimal values for the model parameters. Whereas this approach usually produces overfitting and suboptimal generalization performance, carrying out the Bayesian program of computing the full posterior distributions over the parameters remains a difficult problem. Moreover, learning the structure of models with latent variables, for which the Bayesian approach is crucial, is yet a harder problem. In this paper I present the Variational Bayes framework, which provides a solution to these problems. This approach approximates full posterior distributions over model parameters and structures, as well as latent variables, in an analytical manner without resorting to sampling methods. Unlike in the Laplace approximation, these posteriors are generally non-Gaussian and no Hessian needs to be computed. The resulting algorithm generalizes the standard Expectation Maximization algorithm, and its convergence is guaranteed. I demonstrate that this algorithm can be applied to a large class of models in several domains, including unsupervised clustering and blind source separation.

Appears in Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence (UAI1999)

References in corpus (1)

• The Bayesian Structural EM Algorithm

Cited by in corpus (17)

• Distributed Variational Bayesian Algorithms Over Sensor Networks
• Stochastic Variational Inference
• Fast Inference for Intractable Likelihood Problems using Variational Bayes
• Quantum Annealing for Variational Bayes Inference
• Convergence and asymptotic normality of variational Bayesian approximations for exponential family models with missing values
• Rebuilding Factorized Information Criterion: Asymptotically Accurate Marginal Likelihood
• Variational inference based on a subclass of closed skew normals
• Online reinforcement learning with sparse rewards through an active inference capsule
• Variational Bayesian Inference For A Scale Mixture Of Normal Distributions Handling Missing Data
• A Factorial Mixture Prior for Compositional Deep Generative Models
• Approximating Incomplete Kernel Matrices by the em Algorithm
• Gaussian variational approximation with a factor covariance structure
• Empirical Bayes Linked Matrix Decomposition
• Semisupervised hyperspectral image unmixing using a variational Bayes algorithm
• Hidden Markov Models Applied To Intraday Momentum Trading With Side Information
• Incorporating prior information and borrowing information in high-dimensional sparse regression using the horseshoe and variational Bayes
• Multi-Decoder RNN Autoencoder Based on Variational Bayes Method