The Renyi Gaussian Process: Towards Improved Generalization
arXiv:1910.06990 · doi:10.1080/24725854.2023.2219468
Abstract
We introduce an alternative closed form lower bound on the Gaussian process () likelihood based on the Rényi -divergence. This new lower bound can be viewed as a convex combination of the Nyström approximation and the exact . The key advantage of this bound, is its capability to control and tune the enforced regularization on the model and thus is a generalization of the traditional variational regression. From a theoretical perspective, we provide the convergence rate and risk bound for inference using our proposed approach. Experiments on real data show that the proposed algorithm may be able to deliver improvement over several inference methods.
References in corpus (29)
- Practical Bayesian Optimization of Machine Learning Algorithms
- Variational Inference: A Review for Statisticians
- Neural Tangent Kernel: Convergence and Generalization in Neural Networks
- Variational Inference with Normalizing Flows
- GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration
- Gaussian Processes for Big Data
- Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm
- Deep Gaussian Processes
- BayesOpt: A Bayesian Optimization Library for Nonlinear Optimization, Experimental Design and Bandits
- Generalization in Deep Learning
- Scaling Limits of Wide Neural Networks with Weight Sharing: Gaussian Process Behavior, Gradient Independence, and Neural Tangent Kernel Derivation
- Permutation and Grouping Methods for Sharpening Gaussian Process Approximations
- Deep Gaussian Processes for Regression using Approximate Expectation Propagation
- Sharpness-Aware Minimization for Efficiently Improving Generalization
- When Gaussian Process Meets Big Data: A Review of Scalable GPs
- Gaussian Process Behaviour in Wide Deep Neural Networks
- Bayesian Inference and Learning in Gaussian Process State-Space Models with Particle MCMC
- Rates of Convergence for Sparse Variational Gaussian Process Regression
- A Unifying Framework for Gaussian Process Pseudo-Point Approximations using Power Expectation Propagation
- Product Kernel Interpolation for Scalable Gaussian Processes
- On Bayes Risk Lower Bounds
- Approximate Inference for Fully Bayesian Gaussian Process Regression
- Variational Inference of Joint Models using Multivariate Gaussian Convolution Processes
- Alpha-Beta Divergence For Variational Inference
- Robust Experimental Designs for Model Calibration
- Space-Filling Designs for Robustness Experiments
- Blitzkriging: Kronecker-structured Stochastic Gaussian Processes
- Preconditioning via Diagonal Scaling
- Scalable Gaussian Processes with Billions of Inducing Inputs via Tensor Train Decomposition