Weakly-supervised Multi-output Regression via Correlated Gaussian Processes

Abstract

Multi-output regression seeks to borrow strength and leverage commonalities across different but related outputs in order to enhance learning and prediction accuracy. A fundamental assumption is that the output/group membership labels for all observations are known. This assumption is often violated in real applications. For instance, in healthcare datasets, sensitive attributes such as ethnicity are often missing or unreported. To this end, we introduce a weakly-supervised multi-output model based on dependent Gaussian processes. Our approach is able to leverage data without complete group labels or possibly only prior belief on group memberships to enhance accuracy across all outputs. Through intensive simulations and case studies on an Insulin, Testosterone and Bodyfat dataset, we show that our model excels in multi-output settings with missing labels, while being competitive in traditional fully labeled settings. We end by highlighting the possible use of our approach in fair inference and sequential decision-making.

References in corpus (10)

• Variational Inference: A Review for Statisticians
• Gaussian Processes for Big Data
• Cross-Covariance Functions for Multivariate Geostatistics
• Composite Gaussian process models for emulating expensive functions
• Parallel Bayesian Global Optimization of Expensive Functions
• Semi-supervised Deep Kernel Learning: Regression with Unlabeled Data by Minimizing Predictive Variance
• Minimizing Negative Transfer of Knowledge in Multivariate Gaussian Processes: A Scalable and Regularized Approach
• Variational Inference of Joint Models using Multivariate Gaussian Convolution Processes
• Fully Scalable Gaussian Processes using Subspace Inducing Inputs
• Regression Analysis of Dependent Binary Data for Estimating Disease Etiology from Case-Control Studies