Random Feature Expansions for Deep Gaussian Processes

Abstract

The composition of multiple Gaussian Processes as a Deep Gaussian Process (DGP) enables a deep probabilistic nonparametric approach to flexibly tackle complex machine learning problems with sound quantification of uncertainty. Existing inference approaches for DGP models have limited scalability and are notoriously cumbersome to construct. In this work, we introduce a novel formulation of DGPs based on random feature expansions that we train using stochastic variational inference. This yields a practical learning framework which significantly advances the state-of-the-art in inference for DGPs, and enables accurate quantification of uncertainty. We extensively showcase the scalability and performance of our proposal on several datasets with up to 8 million observations, and various DGP architectures with up to 30 hidden layers.

References in corpus (2)

• Scalable Variational Gaussian Process Classification
• MCMC for Variationally Sparse Gaussian Processes

Cited by in corpus (8)

• Adversarial Examples, Uncertainty, and Transfer Testing Robustness in Gaussian Process Hybrid Deep Networks
• Deep Gaussian Processes for Multi-fidelity Modeling
• Robust Deep Gaussian Processes
• Neural Likelihoods for Multi-Output Gaussian Processes
• Probabilistic Modeling for Novelty Detection with Applications to Fraud Identification
• Modeling and inference of spatio-temporal protein dynamics across brain networks
• Efficient Deep Gaussian Process Models for Variable-Sized Input
• Scalable Grouped Gaussian Processes via Direct Cholesky Functional Representations