Neural Likelihoods for Multi-Output Gaussian Processes

Abstract

We construct flexible likelihoods for multi-output Gaussian process models that leverage neural networks as components. We make use of sparse variational inference methods to enable scalable approximate inference for the resulting class of models. An attractive feature of these models is that they can admit analytic predictive means even when the likelihood is non-linear and the predictive distributions are non-Gaussian. We validate the modeling potential of these models in a variety of experiments in both the supervised and unsupervised setting. We demonstrate that the flexibility of these `neural' likelihoods can improve prediction quality as compared to simpler Gaussian process models and that neural likelihoods can be readily combined with a variety of underlying Gaussian process models, including deep Gaussian processes.

16 pages

References in corpus (12)

• Adam: A Method for Stochastic Optimization
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• Weight Uncertainty in Neural Networks
• Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks
• GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration
• Gaussian Process Kernels for Pattern Discovery and Extrapolation
• Random Feature Expansions for Deep Gaussian Processes
• Variational Auto-encoded Deep Gaussian Processes
• Conditional Neural Processes
• Recurrent Gaussian Processes
• Variational Implicit Processes
• Closed Form Variational Objectives For Bayesian Neural Networks with a Single Hidden Layer