Parallel integrative learning for large-scale multi-response regression with incomplete outcomes

Abstract

Multi-task learning is increasingly used to investigate the association structure between multiple responses and a single set of predictor variables in many applications. In the era of big data, the coexistence of incomplete outcomes, large number of responses, and high dimensionality in predictors poses unprecedented challenges in estimation, prediction, and computation. In this paper, we propose a scalable and computationally efficient procedure, called PEER, for large-scale multi-response regression with incomplete outcomes, where both the numbers of responses and predictors can be high-dimensional. Motivated by sparse factor regression, we convert the multi-response regression into a set of univariate-response regressions, which can be efficiently implemented in parallel. Under some mild regularity conditions, we show that PEER enjoys nice sampling properties including consistency in estimation, prediction, and variable selection. Extensive simulation studies show that our proposal compares favorably with several existing methods in estimation accuracy, variable selection, and computation efficiency.

32 pages

References in corpus (26)

• Rejoinder to "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Least Angle Regression
• Nearly unbiased variable selection under minimax concave penalty
• Simultaneous analysis of Lasso and Dantzig selector
• Pathwise coordinate optimization
• Coordinate descent algorithms for lasso penalized regression
• High-dimensional generalized linear models and the lasso
• Tuning parameter selection in high dimensional penalized likelihood
• A unified approach to model selection and sparse recovery using regularized least squares
• Sparsity oracle inequalities for the Lasso
• The Dantzig selector: Statistical estimation when $p$ is much larger than $n$
• Optimal selection of reduced rank estimators of high-dimensional matrices
• Noisy low-rank matrix completion with general sampling distribution
• Joint variable and rank selection for parsimonious estimation of high-dimensional matrices
• Asymptotic equivalence of regularization methods in thresholded parameter space
• Low Rank Matrix Completion with Exponential Family Noise
• Optimal designs for multi-response generalized linear models with applications in thermal spraying
• Multivariate Functional Regression via Nested Reduced-Rank Regularization

Cited by in corpus (1)

• Simultaneous Heterogeneity and Reduced-rank Learning for Multivariate Response Regression