Slice sampling covariance hyperparameters of latent Gaussian models

Abstract

The Gaussian process (GP) is a popular way to specify dependencies between random variables in a probabilistic model. In the Bayesian framework the covariance structure can be specified using unknown hyperparameters. Integrating over these hyperparameters considers different possible explanations for the data when making predictions. This integration is often performed using Markov chain Monte Carlo (MCMC) sampling. However, with non-Gaussian observations standard hyperparameter sampling approaches require careful tuning and may converge slowly. In this paper we present a slice sampling approach that requires little tuning while mixing well in both strong- and weak-data regimes.

9 pages, 4 figures, 4 algorithms. Minor corrections to previous version. This version to appear in Advances in Neural Information Processing Systems (NIPS) 23, 2010

References in corpus (2)

• Expectation Propagation for approximate Bayesian inference
• Elliptical slice sampling

Cited by in corpus (37)

• Practical Bayesian Optimization of Machine Learning Algorithms
• Nonparametric Bayes dynamic modeling of relational data
• Input Warping for Bayesian Optimization of Non-stationary Functions
• Predictive Entropy Search for Multi-objective Bayesian Optimization with Constraints
• MCMC for Variationally Sparse Gaussian Processes
• Bayesian leave-one-out cross-validation approximations for Gaussian latent variable models
• Raiders of the Lost Architecture: Kernels for Bayesian Optimization in Conditional Parameter Spaces
• Probabilistic prediction of neurological disorders with a statistical assessment of neuroimaging data modalities
• Efficient acquisition rules for model-based approximate Bayesian computation
• Kernel Adaptive Metropolis-Hastings
• Approximate Inference for Fully Bayesian Gaussian Process Regression
• Machine Learning based optimization for interval uncertainty propagation
• Scalable Exact Inference in Multi-Output Gaussian Processes
• Bayesian calibration of traffic flow fundamental diagrams using Gaussian processes
• Enabling scalable stochastic gradient-based inference for Gaussian processes by employing the Unbiased LInear System SolvEr (ULISSE)
• Sequential Gaussian Processes for Online Learning of Nonstationary Functions
• GPatt: Fast Multidimensional Pattern Extrapolation with Gaussian Processes
• Bayesian Optimisation for Active Monitoring of Air Pollution
• Optimal Reinforcement Learning for Gaussian Systems
• Neutron Transmission Strain Tomography for Non-Constant Stress-Free Lattice Spacing
• Bayesian Optimization for Policy Search via Online-Offline Experimentation
• Nonparametric Bayesian Density Modeling with Gaussian Processes
• How Bayesian Should Bayesian Optimisation Be?
• Sparse Orthogonal Variational Inference for Gaussian Processes
• Scalable iterative methods for sampling from massive Gaussian random vectors
• Ensemble Kalman Filtering for Online Gaussian Process Regression and Learning
• Marginalised Gaussian Processes with Nested Sampling
• Scalable Spatiotemporally Varying Coefficient Modelling with Bayesian Kernelized Tensor Regression
• Transitional annealed adaptive slice sampling for Gaussian process hyper-parameter estimation
• Space and circular time log Gaussian Cox processes with application to crime event data
• Bayesian prediction of jumps in large panels of time series data
• Analyzing Car Thefts and Recoveries with Connections to Modeling Origin-Destination Point Patterns
• A Probabilistic Approach to Nonparametric Local Volatility
• Learning Insulin-Glucose Dynamics in the Wild
• Hierarchical Inducing Point Gaussian Process for Inter-domain Observations
• Adaptive Multiple Importance Sampling for Gaussian Processes
• Validating Gaussian Process Models with Simulation-Based Calibration