Gaussian Processes for Big Data

Abstract

We introduce stochastic variational inference for Gaussian process models. This enables the application of Gaussian process (GP) models to data sets containing millions of data points. We show how GPs can be vari- ationally decomposed to depend on a set of globally relevant inducing variables which factorize the model in the necessary manner to perform variational inference. Our ap- proach is readily extended to models with non-Gaussian likelihoods and latent variable models based around Gaussian processes. We demonstrate the approach on a simple toy problem and two real world data sets.

Appears in Proceedings of the Twenty-Ninth Conference on Uncertainty in Artificial Intelligence (UAI2013)

References in corpus (1)

• Fast Variational Inference in the Conjugate Exponential Family

Cited by in corpus (158)

• Hidden Physics Models: Machine Learning of Nonlinear Partial Differential Equations
• Machine Learning of Linear Differential Equations using Gaussian Processes
• Scalable Bayesian Optimization Using Deep Neural Networks
• Scalable Variational Gaussian Process Classification
• Inferring solutions of differential equations using noisy multi-fidelity data
• Kernel Interpolation for Scalable Structured Gaussian Processes (KISS-GP)
• Meta Reinforcement Learning with Latent Variable Gaussian Processes
• When Gaussian Process Meets Big Data: A Review of Scalable GPs
• Functional Regularisation for Continual Learning with Gaussian Processes
• Multi-Fidelity Cost-Aware Bayesian Optimization
• A Framework for Interdomain and Multioutput Gaussian Processes
• Dropout as a Bayesian Approximation: Appendix
• Doubly Stochastic Variational Inference for Deep Gaussian Processes
• Nested Kriging predictions for datasets with large number of observations
• Heterogeneous Multi-output Gaussian Process Prediction
• Rates of Convergence for Sparse Variational Gaussian Process Regression
• A Survey of Optimization Methods from a Machine Learning Perspective
• Grey-box models for wave loading prediction
• Deep Gaussian Processes for Multi-fidelity Modeling
• Deep Gaussian Processes for Biogeophysical Parameter Retrieval and Model Inversion
• Variational Bayesian Unlearning
• Exact Gaussian Processes on a Million Data Points
• Gaussian Process Conditional Density Estimation
• Improving the Gaussian Process Sparse Spectrum Approximation by Representing Uncertainty in Frequency Inputs
• Bayesian Deep Ensembles via the Neural Tangent Kernel
• Multioutput Gaussian Processes with Functional Data: A Study on Coastal Flood Hazard Assessment
• Kernel methods through the roof: handling billions of points efficiently
• A Survey of Bayesian Statistical Approaches for Big Data
• Gaussian Process Molecule Property Prediction with FlowMO
• Advances in Variational Inference
• Variational Bayesian Approximation of Inverse Problems using Sparse Precision Matrices
• Multi-class Gaussian Process Classification with Noisy Inputs
• Comparing Machine Learning and Interpolation Methods for Loop-Level Calculations
• Automated Machine Learning on Big Data using Stochastic Algorithm Tuning
• Early Detection of COVID-19 Hotspots Using Spatio-Temporal Data
• Gaussian Process Prior Variational Autoencoders
• Multi-resolution Multi-task Gaussian Processes
• Algorithmic Linearly Constrained Gaussian Processes
• Deeper Connections between Neural Networks and Gaussian Processes Speed-up Active Learning
• Uncertainty Quantification for Data-driven Turbulence Modelling with Mondrian Forests
• Dynamic Decomposition of Spatiotemporal Neural Signals
• Enforcing Boundary Conditions on Physical Fields in Bayesian Inversion
• Stochastic stiffness identification and response estimation of Timoshenko beams via physics-informed Gaussian processes
• Latent Gaussian Processes for Distribution Estimation of Multivariate Categorical Data
• Deep Gaussian Process-Based Bayesian Inference for Contaminant Source Localization
• Sampling-Free Variational Inference of Bayesian Neural Networks by Variance Backpropagation
• Convergence of Sparse Variational Inference in Gaussian Processes Regression
• GP-select: Accelerating EM using adaptive subspace preselection
• Scalable Exact Inference in Multi-Output Gaussian Processes
• Deep Gaussian Processes: A Survey
• Are we Forgetting about Compositional Optimisers in Bayesian Optimisation?
• Quantifying Point-Prediction Uncertainty in Neural Networks via Residual Estimation with an I/O Kernel
• Improving Robustness and Uncertainty Modelling in Neural Ordinary Differential Equations
• Efficiently Sampling Functions from Gaussian Process Posteriors
• A Tutorial on Sparse Gaussian Processes and Variational Inference
• GPatt: Fast Multidimensional Pattern Extrapolation with Gaussian Processes
• Sequential Gaussian Processes for Online Learning of Nonstationary Functions
• Fast Matrix Square Roots with Applications to Gaussian Processes and Bayesian Optimization
• Scalable Gaussian Processes on Discrete Domains
• Composite likelihood estimation for a gaussian process under fixed domain asymptotics
• Smoothed Gradients for Stochastic Variational Inference
• Gaussian Process Random Fields
• Scaling Gaussian Processes with Derivative Information Using Variational Inference
• Transforming Gaussian Processes With Normalizing Flows
• Multi-Class Gaussian Process Classification Made Conjugate: Efficient Inference via Data Augmentation
• Embarrassingly Parallel Inference for Gaussian Processes
• Continual Multi-task Gaussian Processes
• Probabilistic Active Meta-Learning
• Robust Deep Gaussian Processes
• PI3NN: Out-of-distribution-aware prediction intervals from three neural networks
• Large-scale Heteroscedastic Regression via Gaussian Process
• Probabilistic selection of inducing points in sparse Gaussian processes
• Stochastic Variational Inference for Bayesian Sparse Gaussian Process Regression
• EigenGP: Gaussian Process Models with Adaptive Eigenfunctions
• Theoretical characterization of uncertainty in high-dimensional linear classification
• Ordinal Bayesian Optimisation
• Variational Orthogonal Features
• Low Emission Building Control with Zero-Shot Reinforcement Learning
• A Domain-Shrinking based Bayesian Optimization Algorithm with Order-Optimal Regret Performance
• Multi-task Learning for Aggregated Data using Gaussian Processes
• Parallel and Limited Data Voice Conversion Using Stochastic Variational Deep Kernel Learning
• Weakly-supervised Multi-output Regression via Correlated Gaussian Processes
• Molecular Energy Learning Using Alternative Blackbox Matrix-Matrix Multiplication Algorithm for Exact Gaussian Process
• Optimal Order Simple Regret for Gaussian Process Bandits
• Compositional uncertainty in deep Gaussian processes
• Fast increased fidelity approximate Gibbs samplers for Bayesian Gaussian process regression
• Pathwise Conditioning of Gaussian Processes
• Aggregating Dependent Gaussian Experts in Local Approximation
• Spatio-Temporal Variational Gaussian Processes
• Bayesian Image Classification with Deep Convolutional Gaussian Processes
• Deep Probabilistic Time Series Forecasting using Augmented Recurrent Input for Dynamic Systems
• Variational Bayesian surrogate modelling with application to robust design optimisation
• Modular Gaussian Processes for Transfer Learning
• Efficient Level-Crossing Probability Calculation for Gaussian Process Modeled Data
• Longitudinal Variational Autoencoder
• Recursive Estimation for Sparse Gaussian Process Regression
• Localising Faster: Efficient and precise lidar-based robot localisation in large-scale environments
• Kernel Interpolation for Scalable Online Gaussian Processes
• Scalable Thompson Sampling using Sparse Gaussian Process Models
• Deep Sigma Point Processes
• Uncertainty-Aware Time-to-Event Prediction using Deep Kernel Accelerated Failure Time Models
• Stein Variational Gaussian Processes
• Sequential Deconfounding for Causal Inference with Unobserved Confounders
• Optimal Design Emulators: A Point Process Approach
• Bias-Free Scalable Gaussian Processes via Randomized Truncations
• Matérn Gaussian Processes on Graphs
• Scalable Gaussian Process Inference with Finite-data Mean and Variance Guarantees
• Variational Auto-Regressive Gaussian Processes for Continual Learning
• Inter-domain Deep Gaussian Processes
• Beyond the Mean-Field: Structured Deep Gaussian Processes Improve the Predictive Uncertainties
• SKIing on Simplices: Kernel Interpolation on the Permutohedral Lattice for Scalable Gaussian Processes
• Efficient Transfer Bayesian Optimization with Auxiliary Information
• Scalable Gaussian Process Classification with Additive Noise for Various Likelihoods
• Uncertainty-aware Remaining Useful Life predictor
• Scalable Bayesian dynamic covariance modeling with variational Wishart and inverse Wishart processes
• Quadruply Stochastic Gaussian Processes
• Deep Latent-Variable Kernel Learning
• Variational bridge constructs for approximate Gaussian process regression
• Scalable Gaussian Process Regression for Kernels with a Non-Stationary Phase
• Deep Neural Networks as Point Estimates for Deep Gaussian Processes
• Stochastic Collapsed Variational Inference for Structured Gaussian Process Regression Network
• Quantifying Predictive Uncertainty in Medical Image Analysis with Deep Kernel Learning
• End-to-End Probabilistic Inference for Nonstationary Audio Analysis
• Gaussian Experts Selection using Graphical Models
• Input Dependent Sparse Gaussian Processes
• Online Binary Space Partitioning Forests
• Learning Compositional Sparse Gaussian Processes with a Shrinkage Prior
• State-space deep Gaussian processes with applications
• The Renyi Gaussian Process: Towards Improved Generalization
• Scalable Variational Gaussian Processes via Harmonic Kernel Decomposition
• Learning While Navigating: A Practical System Based on Variational Gaussian Process State-Space Model and Smartphone Sensory Data
• Conditionally Independent Multiresolution Gaussian Processes
• GP-ConvCNP: Better Generalization for Convolutional Conditional Neural Processes on Time Series Data
• Adaptive Cholesky Gaussian Processes
• Searching in the Forest for Local Bayesian Optimization
• Self-explaining variational posterior distributions for Gaussian Process models
• Gaussian Processes to speed up MCMC with automatic exploratory-exploitation effect
• Large-Scale Learning with Fourier Features and Tensor Decompositions
• Hierarchical Inducing Point Gaussian Process for Inter-domain Observations
• Sparse Algorithms for Markovian Gaussian Processes
• Sparse within Sparse Gaussian Processes using Neighbor Information
• Beyond Marginal Uncertainty: How Accurately can Bayesian Regression Models Estimate Posterior Predictive Correlations?
• Direct loss minimization algorithms for sparse Gaussian processes
• Linear-time inference for Gaussian Processes on one dimension
• Healing Products of Gaussian Processes
• On MCMC for variationally sparse Gaussian processes: A pseudo-marginal approach
• Regularized Sparse Gaussian Processes
• Mixtures of Gaussian Processes for regression under multiple prior distributions
• A Probabilistic Assessment of the COVID-19 Lockdown on Air Quality in the UK
• Orthogonally Decoupled Variational Fourier Features
• Modulating Scalable Gaussian Processes for Expressive Statistical Learning
• Recyclable Gaussian Processes
• Statistically and Computationally Efficient Variance Estimator for Kernel Ridge Regression
• Scalable Hyperparameter Optimization with Lazy Gaussian Processes
• Probabilistic DAG Search
• Scalable Grouped Gaussian Processes via Direct Cholesky Functional Representations
• Online Anomaly Detection with Sparse Gaussian Processes
• Interpolation and extrapolation of global potential energy surfaces for polyatomic systems by Gaussian processes with composite kernels