Aggregating Dependent Gaussian Experts in Local Approximation

Abstract

Distributed Gaussian processes (DGPs) are prominent local approximation methods to scale Gaussian processes (GPs) to large datasets. Instead of a global estimation, they train local experts by dividing the training set into subsets, thus reducing the time complexity. This strategy is based on the conditional independence assumption, which basically means that there is a perfect diversity between the local experts. In practice, however, this assumption is often violated, and the aggregation of experts leads to sub-optimal and inconsistent solutions. In this paper, we propose a novel approach for aggregating the Gaussian experts by detecting strong violations of conditional independence. The dependency between experts is determined by using a Gaussian graphical model, which yields the precision matrix. The precision matrix encodes conditional dependencies between experts and is used to detect strongly dependent experts and construct an improved aggregation. Using both synthetic and real datasets, our experimental evaluations illustrate that our new method outperforms other state-of-the-art (SOTA) DGP approaches while being substantially more time-efficient than SOTA approaches, which build on independent experts.

arXiv comment8 pages

References in corpus (9)

• High-dimensional graphs and variable selection with the Lasso
• Gaussian Processes for Data-Efficient Learning in Robotics and Control
• Gaussian Processes for Big Data
• Ranking and combining multiple predictors without labeled data
• Generalized Product of Experts for Automatic and Principled Fusion of Gaussian Process Predictions
• Nested Kriging predictions for datasets with large number of observations
• An asymptotic analysis of distributed nonparametric methods
• Gaussian Graphical Models: An Algebraic and Geometric Perspective
• Properties and comparison of some Kriging sub-model aggregation methods

Cited by in corpus (1)

• Gaussian Experts Selection using Graphical Models