Deep Gaussian Processes
arXiv:1211.0358
Abstract
In this paper we introduce deep Gaussian process (GP) models. Deep GPs are a deep belief network based on Gaussian process mappings. The data is modeled as the output of a multivariate GP. The inputs to that Gaussian process are then governed by another GP. A single layer model is equivalent to a standard GP or the GP latent variable model (GP-LVM). We perform inference in the model by approximate variational marginalization. This results in a strict lower bound on the marginal likelihood of the model which we use for model selection (number of layers and nodes per layer). Deep belief networks are typically applied to relatively large data sets using stochastic gradient descent for optimization. Our fully Bayesian treatment allows for the application of deep models even when data is scarce. Model selection by our variational bound shows that a five layer hierarchy is justified even when modelling a digit data set containing only 150 examples.
9 pages, 8 figures. Appearing in Proceedings of the 16th International Conference on Artificial Intelligence and Statistics (AISTATS) 2013
References in corpus (2)
Cited by in corpus (43)
- A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
- Time Series Forecasting With Deep Learning: A Survey
- Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems
- Bayesian-Deep-Learning Estimation of Earthquake Location from Single-Station Observations
- Probabilistic neural networks for fluid flow surrogate modeling and data recovery
- A Bayesian approach to constrained single- and multi-objective optimization
- How priors of initial hyperparameters affect Gaussian process regression models
- Probabilistic Forecasting of Sensory Data with Generative Adversarial Networks - ForGAN
- Ship Performance Monitoring using Machine-learning
- DAC-h3: A Proactive Robot Cognitive Architecture to Acquire and Express Knowledge About the World and the Self
- Survey on Machine Learning for Traffic-Driven Service Provisioning in Optical Networks
- Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety
- Global Optimization of Gaussian processes
- Structured Bayesian Gaussian process latent variable model: applications to data-driven dimensionality reduction and high-dimensional inversion
- Deep Gaussian Processes for Biogeophysical Parameter Retrieval and Model Inversion
- A Latent Restoring Force Approach to Nonlinear System Identification
- A Survey of Bayesian Statistical Approaches for Big Data
- Gaussian Process Models with Parallelization and GPU acceleration
- Variational Inference of Joint Models using Multivariate Gaussian Convolution Processes
- Deep Gaussian Process-Based Bayesian Inference for Contaminant Source Localization
- Bayesian Optimization using Deep Gaussian Processes
- Gradient and Uncertainty Enhanced Sequential Sampling for Global Fit
- Measuring Uncertainty in Signal Fingerprinting with Gaussian Processes Going Deep
- Interpretable Uncertainty Quantification in AI for HEP
- How Bayesian Should Bayesian Optimisation Be?
- Deep Gaussian Processes for geophysical parameter retrieval
- Multi-fidelity data fusion for the approximation of scalar functions with low intrinsic dimensionality through active subspaces
- Towards Expressive Priors for Bayesian Neural Networks: Poisson Process Radial Basis Function Networks
- Data-Driven Topology Optimization with Multiclass Microstructures using Latent Variable Gaussian Process
- Learning of Tree-Structured Gaussian Graphical Models on Distributed Data under Communication Constraints
- On Sparse variational methods and the Kullback-Leibler divergence between stochastic processes
- Bayesian Alignments of Warped Multi-Output Gaussian Processes
- Deep Recurrent Gaussian Process with Variational Sparse Spectrum Approximation
- SDG-L: A Semiparametric Deep Gaussian Process based Framework for Battery Capacity Prediction
- Deep Gaussian Processes for Air Quality Inference
- Energy-Based Processes for Exchangeable Data
- Probabilistic partition of unity networks for high-dimensional regression problems
- On Neural Networks as Infinite Tree-Structured Probabilistic Graphical Models
- The Renyi Gaussian Process: Towards Improved Generalization
- Posterior contraction rates for constrained deep Gaussian processes in density estimation and classication
- Relaxed Gaussian process interpolation: a goal-oriented approach to Bayesian optimization
- MFPC-Net: Multi-fidelity Physics-Constrained Neural Process
- Global Optimisation of Black-Box Functions with Generative Models in the Wasserstein Space