Nested Variational Compression in Deep Gaussian Processes

Abstract

Deep Gaussian processes provide a flexible approach to probabilistic modelling of data using either supervised or unsupervised learning. For tractable inference approximations to the marginal likelihood of the model must be made. The original approach to approximate inference in these models used variational compression to allow for approximate variational marginalization of the hidden variables leading to a lower bound on the marginal likelihood of the model [Damianou and Lawrence, 2013]. In this paper we extend this idea with a nested variational compression. The resulting lower bound on the likelihood can be easily parallelized or adapted for stochastic variational inference.

References in corpus (2)

• Avoiding pathologies in very deep networks
• Stochastic Variational Inference

Cited by in corpus (23)

• Deep Neural Networks as Gaussian Processes
• Gradient-based Hyperparameter Optimization through Reversible Learning
• Deep Gaussian Processes for Regression using Approximate Expectation Propagation
• Variational Auto-encoded Deep Gaussian Processes
• Deep Gaussian Processes with Convolutional Kernels
• Recent advances in deep learning theory
• Exploiting gradients and Hessians in Bayesian optimization and Bayesian quadrature
• Early Stopping is Nonparametric Variational Inference
• On the expected behaviour of noise regularised deep neural networks as Gaussian processes
• Robust Deep Gaussian Processes
• Large-scale Heteroscedastic Regression via Gaussian Process
• Probabilistic selection of inducing points in sparse Gaussian processes
• Multi-view Learning as a Nonparametric Nonlinear Inter-Battery Factor Analysis
• Wide Neural Networks with Bottlenecks are Deep Gaussian Processes
• Compositional uncertainty in deep Gaussian processes
• Bayesian Alignments of Warped Multi-Output Gaussian Processes
• Building Bayesian Neural Networks with Blocks: On Structure, Interpretability and Uncertainty
• Deep Recurrent Gaussian Process with Variational Sparse Spectrum Approximation
• Uncertainty Quantification of Darcy Flow through Porous Media using Deep Gaussian Process
• DeepCoder: Semi-parametric Variational Autoencoders for Automatic Facial Action Coding
• Inter-domain Deep Gaussian Processes
• Deep Neural Networks as Point Estimates for Deep Gaussian Processes
• Convolutional Normalizing Flows for Deep Gaussian Processes