Student-t Processes as Alternatives to Gaussian Processes

Abstract

We investigate the Student-t process as an alternative to the Gaussian process as a nonparametric prior over functions. We derive closed form expressions for the marginal likelihood and predictive distribution of a Student-t process, by integrating away an inverse Wishart process prior over the covariance kernel of a Gaussian process model. We show surprising equivalences between different hierarchical Gaussian process models leading to Student-t processes, and derive a new sampling scheme for the inverse Wishart process, which helps elucidate these equivalences. Overall, we show that a Student-t process can retain the attractive properties of a Gaussian process -- a nonparametric representation, analytic marginal and predictive distributions, and easy model selection through covariance kernels -- but has enhanced flexibility, and predictive covariances that, unlike a Gaussian process, explicitly depend on the values of training observations. We verify empirically that a Student-t process is especially useful in situations where there are changes in covariance structure, or in applications like Bayesian optimization, where accurate predictive covariances are critical for good performance. These advantages come at no additional computational cost over Gaussian processes.

13 pages, 6 figures, 1 table. To appear in "The Seventeenth International Conference on Artificial Intelligence and Statistics (AISTATS), 2014."

References in corpus (2)

• Practical Bayesian Optimization of Machine Learning Algorithms
• A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning

Cited by in corpus (33)

• Dealing with Integer-valued Variables in Bayesian Optimization with Gaussian Processes
• Dealing with Categorical and Integer-valued Variables in Bayesian Optimization with Gaussian Processes
• Multivariate Gaussian and Student$-t$ Process Regression for Multi-output Prediction
• Bayesian Optimization for Likelihood-Free Inference of Simulator-Based Statistical Models
• Predictive Entropy Search for Multi-objective Bayesian Optimization with Constraints
• t-Soft Update of Target Network for Deep Reinforcement Learning
• Parallel Predictive Entropy Search for Batch Global Optimization of Expensive Objective Functions
• Efficient leave-one-out cross-validation for Bayesian non-factorized normal and Student-t models
• Bayesian Multinomial Logistic Normal Models through Marginally Latent Matrix-T Processes
• A Fast, 2D Gaussian Process Method Based on Celerite: Applications to Transiting Exoplanet Discovery and Characterization
• On the average uncertainty for systems with nonlinear coupling
• Kalman Filter Tuning with Bayesian Optimization
• Evaluating Gaussian Process Metamodels and Sequential Designs for Noisy Level Set Estimation
• ProBO: Versatile Bayesian Optimization Using Any Probabilistic Programming Language
• Variational Implicit Processes
• Set Flow: A Permutation Invariant Normalizing Flow
• Correlated Random Measures
• Unfolding with Gaussian Processes
• Extended T-process Regression Models
• Scale Mixtures of Neural Network Gaussian Processes
• Non-smooth Bayesian Optimization in Tuning Problems
• Parameter and Uncertainty Estimation for Dynamical Systems Using Surrogate Stochastic Processes
• Deep kernel processes
• Energy-Based Processes for Exchangeable Data
• Bayesian Quadrature Optimization for Probability Threshold Robustness Measure
• Transport Gaussian Processes for Regression
• Conditional Deep Gaussian Processes: multi-fidelity kernel learning
• Funneled Bayesian Optimization for Design, Tuning and Control of Autonomous Systems
• Robust functional ANOVA model with t-process
• Bayesian Inference Gaussian Process Multiproxy Alignment of Continuous Signals (BIGMACS): Applications for Paleoceanography
• String and Membrane Gaussian Processes
• Comment on "Probabilistic Integration: A Role in Statistical Computation?"
• Partially Observed Exchangeable Modeling