Semi-supervised Deep Kernel Learning: Regression with Unlabeled Data by Minimizing Predictive Variance

Abstract

Large amounts of labeled data are typically required to train deep learning models. For many real-world problems, however, acquiring additional data can be expensive or even impossible. We present semi-supervised deep kernel learning (SSDKL), a semi-supervised regression model based on minimizing predictive variance in the posterior regularization framework. SSDKL combines the hierarchical representation learning of neural networks with the probabilistic modeling capabilities of Gaussian processes. By leveraging unlabeled data, we show improvements on a diverse set of real-world regression tasks over supervised deep kernel learning and semi-supervised methods such as VAT and mean teacher adapted for regression.

arXiv commentIn Proceedings of Neural Information Processing Systems (NeurIPS) 2018

References in corpus (11)

• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
• Semi-Supervised Learning with Ladder Networks
• A DIRT-T Approach to Unsupervised Domain Adaptation
• Distributional Smoothing with Virtual Adversarial Training
• Auxiliary Deep Generative Models
• Kernel Interpolation for Scalable Structured Gaussian Processes (KISS-GP)
• Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning
• Stochastic Variational Deep Kernel Learning
• Amortized Inference Regularization
• Learning Scalable Deep Kernels with Recurrent Structure

Cited by in corpus (12)

• WILDS: A Benchmark of in-the-Wild Distribution Shifts
• Improved Training of Physics-Informed Neural Networks with Model Ensembles
• Twin Neural Network Regression is a Semi-Supervised Regression Algorithm
• Learning deep kernels for exponential family densities
• Learning to Impute: A General Framework for Semi-supervised Learning
• Deep Low-Density Separation for Semi-Supervised Classification
• Weakly-supervised Multi-output Regression via Correlated Gaussian Processes
• Built Year Prediction from Buddha Face with Heterogeneous Labels
• Post-Estimation Smoothing: A Simple Baseline for Learning with Side Information
• Learning Data-adaptive Nonparametric Kernels
• High-Dimensional Bayesian Optimization via Semi-Supervised Learning with Optimized Unlabeled Data Sampling
• Covariate Distribution Aware Meta-learning