Learning Gaussian Processes by Minimizing PAC-Bayesian Generalization Bounds

Abstract

Gaussian Processes (GPs) are a generic modelling tool for supervised learning. While they have been successfully applied on large datasets, their use in safety-critical applications is hindered by the lack of good performance guarantees. To this end, we propose a method to learn GPs and their sparse approximations by directly optimizing a PAC-Bayesian bound on their generalization performance, instead of maximizing the marginal likelihood. Besides its theoretical appeal, we find in our evaluation that our learning method is robust and yields significantly better generalization guarantees than other common GP approaches on several regression benchmark datasets.

arXiv comment11 pages main text, 12 pages appendix. v2: minor changes, new NeurIPS style file. Final camera-ready version submitted to NeurIPS 2018

Cited by in corpus (9)

• How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Review
• Personalized Federated Learning with Gaussian Processes
• Posterior Variance Analysis of Gaussian Processes with Application to Average Learning Curves
• Stochastic Neural Network with Kronecker Flow
• A PAC-Bayes Analysis of Adversarial Robustness
• Learning Partially Known Stochastic Dynamics with Empirical PAC Bayes
• Beyond the Mean-Field: Structured Deep Gaussian Processes Improve the Predictive Uncertainties
• Self-Bounding Majority Vote Learning Algorithms by the Direct Minimization of a Tight PAC-Bayesian C-Bound
• Convolutional Normalizing Flows for Deep Gaussian Processes