Random Feature Expansions for Deep Gaussian Processes

Abstract

The composition of multiple Gaussian Processes as a Deep Gaussian Process (DGP) enables a deep probabilistic nonparametric approach to flexibly tackle complex machine learning problems with sound quantification of uncertainty. Existing inference approaches for DGP models have limited scalability and are notoriously cumbersome to construct. In this work, we introduce a novel formulation of DGPs based on random feature expansions that we train using stochastic variational inference. This yields a practical learning framework which significantly advances the state-of-the-art in inference for DGPs, and enables accurate quantification of uncertainty. We extensively showcase the scalability and performance of our proposal on several datasets with up to 8 million observations, and various DGP architectures with up to 30 hidden layers.

References in corpus (6)

• Auto-Encoding Variational Bayes
• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• Scalable Variational Gaussian Process Classification
• Deep Gaussian Processes for Regression using Approximate Expectation Propagation
• MCMC for Variationally Sparse Gaussian Processes

Cited by in corpus (39)

• Adversarial Examples, Uncertainty, and Transfer Testing Robustness in Gaussian Process Hybrid Deep Networks
• When Gaussian Process Meets Big Data: A Review of Scalable GPs
• Generalized Variational Inference: Three arguments for deriving new Posteriors
• Deep Gaussian Processes for Multi-fidelity Modeling
• The Functional Neural Process
• Kernel methods through the roof: handling billions of points efficiently
• Kernel computations from large-scale random features obtained by Optical Processing Units
• Deep Gaussian Processes with Convolutional Kernels
• Calibrating Deep Convolutional Gaussian Processes
• Fast Kernel Approximations for Latent Force Models and Convolved Multiple-Output Gaussian processes
• Robust Deep Gaussian Processes
• Wide Neural Networks with Bottlenecks are Deep Gaussian Processes
• The Limitations of Large Width in Neural Networks: A Deep Gaussian Process Perspective
• Scalable Thompson Sampling using Sparse Gaussian Process Models
• Walsh-Hadamard Variational Inference for Bayesian Deep Learning
• Building Bayesian Neural Networks with Blocks: On Structure, Interpretability and Uncertainty
• Sparse Spectrum Warped Input Measures for Nonstationary Kernel Learning
• Deep Sigma Point Processes
• A Neural Network Approach for Online Nonlinear Neyman-Pearson Classification
• Inter-domain Deep Gaussian Processes
• Alzheimer's Disease Modelling and Staging through Independent Gaussian Process Analysis of Spatio-Temporal Brain Changes
• Neural Likelihoods for Multi-Output Gaussian Processes
• Probabilistic Modeling for Novelty Detection with Applications to Fraud Identification
• Compositional Modeling of Nonlinear Dynamical Systems with ODE-based Random Features
• Modeling and inference of spatio-temporal protein dynamics across brain networks
• Deep Neural Networks as Point Estimates for Deep Gaussian Processes
• Posterior contraction for deep Gaussian process priors
• PAC-Bayesian Bounds for Deep Gaussian Processes
• Conditional Deep Gaussian Processes: empirical Bayes hyperdata learning
• Scalable Grouped Gaussian Processes via Direct Cholesky Functional Representations
• Utterance-level Sequential Modeling For Deep Gaussian Process Based Speech Synthesis Using Simple Recurrent Unit
• Probabilistic Re-aggregation Algorithm [First Draft]
• Prostate Tissue Grading with Deep Quantum Measurement Ordinal Regression
• Efficient Deep Gaussian Process Models for Variable-Sized Input
• Sparse within Sparse Gaussian Processes using Neighbor Information
• Explaining Deep Tractable Probabilistic Models: The sum-product network case
• Deep Neural-Kernel Machines
• Convolutional Normalizing Flows for Deep Gaussian Processes
• Efficient Approximate Inference with Walsh-Hadamard Variational Inference