Generalized Product of Experts for Automatic and Principled Fusion of Gaussian Process Predictions

Abstract

In this work, we propose a generalized product of experts (gPoE) framework for combining the predictions of multiple probabilistic models. We identify four desirable properties that are important for scalability, expressiveness and robustness, when learning and inferring with a combination of multiple models. Through analysis and experiments, we show that gPoE of Gaussian processes (GP) have these qualities, while no other existing combination schemes satisfy all of them at the same time. The resulting GP-gPoE is highly scalable as individual GP experts can be independently learned in parallel; very expressive as the way experts are combined depends on the input rather than fixed; the combined prediction is still a valid probabilistic model with natural interpretation; and finally robust to unreliable predictions from individual experts.

Modern Nonparametrics 3: Automating the Learning Pipeline workshop at NIPS 2014

Cited by in corpus (33)

• Multimodal Generative Models for Scalable Weakly-Supervised Learning
• Hetero-Modal Variational Encoder-Decoder for Joint Modality Completion and Segmentation
• When Gaussian Process Meets Big Data: A Review of Scalable GPs
• Minimizing Negative Transfer of Knowledge in Multivariate Gaussian Processes: A Scalable and Regularized Approach
• Normative Modeling using Multimodal Variational Autoencoders to Identify Abnormal Brain Structural Patterns in Alzheimer Disease
• Deep Structured Mixtures of Gaussian Processes
• Composite likelihood estimation for a gaussian process under fixed domain asymptotics
• Embarrassingly Parallel Inference for Gaussian Processes
• Fast Deep Mixtures of Gaussian Process Experts
• Gaussian Gated Linear Networks
• Distributed Bayesian Online Learning for Cooperative Manipulation
• Product Kanerva Machines: Factorized Bayesian Memory
• Pillar Networks++: Distributed non-parametric deep and wide networks
• Modular Gaussian Processes for Transfer Learning
• A Simple Framework for Uncertainty in Contrastive Learning
• Aggregating Dependent Gaussian Experts in Local Approximation
• Multi-View Variational Autoencoder for Missing Value Imputation in Untargeted Metabolomics
• Nonparametric Bayesian Aggregation for Massive Data
• Scalable Bayesian Non-linear Matrix Completion
• Bias-Free Scalable Gaussian Processes via Randomized Truncations
• Rumor-robust Decentralized Gaussian Process Learning, Fusion, and Planning for Modeling Multiple Moving Targets
• Gaussian Experts Selection using Graphical Models
• Imitation Learning for Fashion Style Based on Hierarchical Multimodal Representation
• String and Membrane Gaussian Processes
• Recyclable Gaussian Processes
• Healing Products of Gaussian Processes
• Recommending Burgers based on Pizza Preferences: Addressing Data Sparsity with a Product of Experts
• Scalable Bayesian Optimization with Sparse Gaussian Process Models
• Sequential online prediction in the presence of outliers and change points: an instant temporal structure learning approach
• Temporal Data Fusion at the Edge
• Harnessing Low-Fidelity Data to Accelerate Bayesian Optimization via Posterior Regularization
• Composite Gaussian Processes: Scalable Computation and Performance Analysis
• Revisiting Large Scale Distributed Machine Learning