Regression for matrix-valued data via Kronecker products factorization

Abstract

We study the matrix-variate regression problem $Y_i = \sum_{k} β_{1k} X_i β_{2k}^{\top} + E_i$ for $i=1,2\dots ,n$ in the high dimensional regime wherein the response $Y_i$ are matrices whose dimensions $p_1\times p_2$ outgrow both the sample size $n$ and the dimensions $q_1\times q_2$ of the predictor variables $X_i$ i.e., $q_1,q_2\ll n\ll p_1,p_2$. We propose an estimation algorithm, termed KRO-PRO-FAC, for estimating the parameters $\{{\hat{I}}_{1k}^2\}\subset {\Re }^{p_1\times q_1}$ and $\{{\hat{I}}_{2k}^2\}\subset {\Re }^{p_2\times q_2}$ that utilizes the Kronecker product factorization and rearrangement operations from Van Loan and Pitsianis (1993). The KRO-PRO-FAC algorithm is computationally efficient as it does not require estimating the covariance between the entries of the $\{Y_i\}$. We establish perturbation bounds between ${\hat{\hat{I}}}_{1k}^2-{\hat{I}}_{1k}^2$ and ${\hat{\hat{I}}}_{2k}^2-{\hat{I}}_{2k}^2$ in spectral norm for the setting where either the rows of $E_i$ or the columns of $E_i$ are independent sub-Gaussian random vectors. Numerical studies on simulated and real data indicate that our procedure is competitive, in terms of both estimation error and predictive accuracy, compared to other existing methods.