Chi-square and normal inference in high-dimensional multi-task regression

Abstract

The paper proposes chi-square and normal inference methodologies for the unknown coefficient matrix $B^{\ast }$ of size $p\times T$ in a Multi-Task (MT) linear model with $p$ covariates, $T$ tasks and $n$ observations under a row-sparse assumption on $B^{\ast }$. The row-sparsity $s$, dimension $p$ and number of tasks $T$ are allowed to grow with $n$. In the high-dimensional regime $p⋙n$, in order to leverage row-sparsity, the MT Lasso is considered. We build upon the MT Lasso with a de-biasing scheme to correct for the bias induced by the penalty. This scheme requires the introduction of a new data-driven object, coined the interaction matrix, that captures effective correlations between noise vector and residuals on different tasks. This matrix is psd, of size $T\times T$ and can be computed efficiently. The interaction matrix lets us derive asymptotic normal and ${\chi }_T^2$ results under Gaussian design and $\frac{sT+s\log (p/s)}{n}\to 0$ which corresponds to consistency in Frobenius norm. These asymptotic distribution results yield valid confidence intervals for single entries of $B^{\ast }$ and valid confidence ellipsoids for single rows of $B^{\ast }$, for both known and unknown design covariance $\Sigma$. While previous proposals in grouped-variables regression require row-sparsity $s\lesssim \sqrt{n}$ up to constants depending on $T$ and logarithmic factors in $n,p$, the de-biasing scheme using the interaction matrix provides confidence intervals and ${\chi }_T^2$ confidence ellipsoids under the conditions $\min (T^2,{\log }^{8}p)/n\to 0$ and $$\frac{sT+s\log (p/s)+\Vert {\Sigma }^{-1}{e}_j{\Vert }_0\log p}{n}\to 0,\frac{\min (s,\Vert {\Sigma }^{-1}{e}_j{\Vert }_0)}{\sqrt{n}}\sqrt{[T+\log (p/s)]\log p}\to 0,$$ allowing row-sparsity $s⋙\sqrt{n}$ when $\Vert {\Sigma }^{-1}{e}_j{\Vert }_0\sqrt{T}⋘\sqrt{n}$ up to logarithmic factors.

References in corpus (13)

• Scikit-learn: Machine Learning in Python
• Simultaneous analysis of Lasso and Dantzig selector
• Confidence Intervals and Hypothesis Testing for High-Dimensional Regression
• Square-Root Lasso: Pivotal Recovery of Sparse Signals via Conic Programming
• Support union recovery in high-dimensional multivariate regression
• The Lasso with general Gaussian designs with applications to hypothesis testing
• Testability of high-dimensional linear models with non-sparse structures
• De-Biasing The Lasso With Degrees-of-Freedom Adjustment
• Asymptotic Confidence Regions for High-dimensional Structured Sparsity
• Second order Stein: SURE for SURE and other applications in high-dimensional inference
• The noise barrier and the large signal bias of the Lasso and other convex estimators
• Statistical control for spatio-temporal MEG/EEG source imaging with desparsified multi-task Lasso
• Support recovery and sup-norm convergence rates for sparse pivotal estimation

Cited by in corpus (1)

• The EAS approach to variable selection for multivariate response data in high-dimensional settings