GP-select: Accelerating EM using adaptive subspace preselection

Abstract

We propose a nonparametric procedure to achieve fast inference in generative graphical models when the number of latent states is very large. The approach is based on iterative latent variable preselection, where we alternate between learning a 'selection function' to reveal the relevant latent variables, and use this to obtain a compact approximation of the posterior distribution for EM; this can make inference possible where the number of possible latent states is e.g. exponential in the number of latent variables, whereas an exact approach would be computationally unfeasible. We learn the selection function entirely from the observed data and current EM state via Gaussian process regression. This is by contrast with earlier approaches, where selection functions were manually-designed for each problem setting. We show that our approach performs as well as these bespoke selection functions on a wide variety of inference problems: in particular, for the challenging case of a hierarchical model for object localization with occlusion, we achieve results that match a customized state-of-the-art selection method, at a far lower computational cost.

References in corpus (12)

• Fast and Accurate Modeling of Molecular Atomization Energies with Machine Learning
• Scalable Bayesian Optimization Using Deep Neural Networks
• Gaussian Processes for Big Data
• Divide and Conquer Kernel Ridge Regression: A Distributed Algorithm with Minimax Optimal Rates
• A sparse coding model with synaptically local plasticity and spiking neurons can account for the diverse shapes of V1 simple cell receptive fields
• Neural Variational Inference and Learning in Belief Networks
• GPS-ABC: Gaussian Process Surrogate Approximate Bayesian Computation
• Proceedings of the Twenty-Ninth Conference on Uncertainty in Artificial Intelligence (2013)
• Fast Variational Inference in the Conjugate Exponential Family
• Efficient Gradient-Based Inference through Transformations between Bayes Nets and Neural Nets
• Gaussian Process Models with Parallelization and GPU acceleration
• Autonomous Cleaning of Corrupted Scanned Documents - A Generative Modeling Approach

Cited by in corpus (6)

• $k$-means as a variational EM approximation of Gaussian mixture models
• Neural Simpletrons - Minimalistic Directed Generative Networks for Learning with Few Labels
• Truncated Variational Expectation Maximization
• Large Scale Clustering with Variational EM for Gaussian Mixture Models
• Direct Evolutionary Optimization of Variational Autoencoders With Binary Latents
• Evolutionary Variational Optimization of Generative Models