Variational Auto-encoded Deep Gaussian Processes

Abstract

We develop a scalable deep non-parametric generative model by augmenting deep Gaussian processes with a recognition model. Inference is performed in a novel scalable variational framework where the variational posterior distributions are reparametrized through a multilayer perceptron. The key aspect of this reformulation is that it prevents the proliferation of variational parameters which otherwise grow linearly in proportion to the sample size. We derive a new formulation of the variational lower bound that allows us to distribute most of the computation in a way that enables to handle datasets of the size of mainstream deep learning tasks. We show the efficacy of the method on a variety of challenges including deep unsupervised learning and deep Bayesian optimization.

References in corpus (9)

• Improving neural networks by preventing co-adaptation of feature detectors
• Practical Bayesian Optimization of Machine Learning Algorithms
• Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
• Generative Moment Matching Networks
• Better Mixing via Deep Representations
• Nested Variational Compression in Deep Gaussian Processes
• Gaussian Process Models with Parallelization and GPU acceleration
• Semi-described and semi-supervised learning with Gaussian processes
• Additive Kernels for Gaussian Process Modeling

Cited by in corpus (18)

• Deep Gaussian Processes for Regression using Approximate Expectation Propagation
• When Gaussian Process Meets Big Data: A Review of Scalable GPs
• Structured Variational Inference for Simulating Populations of Radio Galaxies
• Bayesian Optimization using Deep Gaussian Processes
• Deep Gaussian Processes: A Survey
• Identification of Gaussian Process State Space Models
• On the expected behaviour of noise regularised deep neural networks as Gaussian processes
• Robust Deep Gaussian Processes
• GPflux: A Library for Deep Gaussian Processes
• Wide Neural Networks with Bottlenecks are Deep Gaussian Processes
• How Deep Are Deep Gaussian Processes?
• A Survey of Techniques All Classifiers Can Learn from Deep Networks: Models, Optimizations, and Regularization
• Building Bayesian Neural Networks with Blocks: On Structure, Interpretability and Uncertainty
• Neural Likelihoods for Multi-Output Gaussian Processes
• Inter-domain Deep Gaussian Processes
• Closed Form Variational Objectives For Bayesian Neural Networks with a Single Hidden Layer
• An Efficient Large-scale Semi-supervised Multi-label Classifier Capable of Handling Missing labels
• Epoch-evolving Gaussian Process Guided Learning