Semi-supervised linear regression: enhancing efficiency and robustness in high dimensions

Abstract

In semi-supervised learning, the prevailing understanding suggests that observing additional unlabeled samples improves estimation accuracy for linear parameters only in the case of model misspecification. In this work, we challenge such a claim and show that additional unlabeled samples are beneficial in high-dimensional settings. Initially focusing on a dense scenario, we introduce robust semi-supervised estimators for the regression coefficient without relying on sparse structures in the population slope. Even when the true underlying model is linear, we show that leveraging information from large-scale unlabeled data helps reduce estimation bias, thereby improving both estimation accuracy and inference robustness. Moreover, we propose semi-supervised methods with further enhanced efficiency in scenarios with a sparse linear slope. The performance of the proposed methods is demonstrated through extensive numerical studies.

References in corpus (10)

• On the "degrees of freedom" of the lasso
• Confidence Intervals and Hypothesis Testing for High-Dimensional Regression
• Degrees of freedom in lasso problems
• Efficient and Adaptive Linear Regression in Semi-Supervised Settings
• Linear Hypothesis Testing in Dense High-Dimensional Linear Models
• Robust and Efficient Semi-Supervised Estimation of Average Treatment Effects with Application to Electronic Health Records Data
• Testability of high-dimensional linear models with non-sparse structures
• A General Framework for Treatment Effect Estimation in Semi-Supervised and High Dimensional Settings
• Empirical Likelihood-Based Estimation and Inference in Randomized Controlled Trials with High-Dimensional Covariates
• Adaptive and Efficient Learning with Blockwise Missing and Semi-Supervised Data