Optimal selection of reduced rank estimators of high-dimensional matrices

Abstract

We introduce a new criterion, the Rank Selection Criterion (RSC), for selecting the optimal reduced rank estimator of the coefficient matrix in multivariate response regression models. The corresponding RSC estimator minimizes the Frobenius norm of the fit plus a regularization term proportional to the number of parameters in the reduced rank model. The rank of the RSC estimator provides a consistent estimator of the rank of the coefficient matrix; in general, the rank of our estimator is a consistent estimate of the effective rank, which we define to be the number of singular values of the target matrix that are appropriately large. The consistency results are valid not only in the classic asymptotic regime, when $n$, the number of responses, and $p$, the number of predictors, stay bounded, and $m$, the number of observations, grows, but also when either, or both, $n$ and $p$ grow, possibly much faster than $m$. We establish minimax optimal bounds on the mean squared errors of our estimators. Our finite sample performance bounds for the RSC estimator show that it achieves the optimal balance between the approximation error and the penalty term. Furthermore, our procedure has very low computational complexity, linear in the number of candidate models, making it particularly appealing for large scale problems. We contrast our estimator with the nuclear norm penalized least squares (NNP) estimator, which has an inherently higher computational complexity than RSC, for multivariate regression models. We show that NNP has estimation properties similar to those of RSC, albeit under stronger conditions. However, it is not as parsimonious as RSC. We offer a simple correction of the NNP estimator which leads to consistent rank estimation.

Published in at http://dx.doi.org/10.1214/11-AOS876 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org) (some typos corrected)

References in corpus (1)

• Estimation of high-dimensional low-rank matrices

Cited by in corpus (18)

• Estimation of high-dimensional low-rank matrices
• Noisy low-rank matrix completion with general sampling distribution
• Joint variable and rank selection for parsimonious estimation of high-dimensional matrices
• On the sample covariance matrix estimator of reduced effective rank population matrices, with applications to fPCA
• Adaptive estimation of the copula correlation matrix for semiparametric elliptical copulas
• Bayesian methods for low-rank matrix estimation: short survey and theoretical study
• Optimal large-scale quantum state tomography with Pauli measurements
• Rank penalized estimation of a quantum system
• Fast estimation of approximate matrix ranks using spectral densities
• Asymptotic equivalence of quantum state tomography and noisy matrix completion
• Spectral thresholding quantum tomography for low rank states
• High dimensional VAR with low rank transition
• Inferring Influence Networks from Longitudinal Bipartite Relational Data
• A reduced-rank approach to predicting multiple binary responses through machine learning
• Parallel integrative learning for large-scale multi-response regression with incomplete outcomes
• Estimators for multivariate allometric regression model
• Low solution rank of the matrix LASSO under RIP with consequences for rank-constrained algorithms
• Simultaneous Heterogeneity and Reduced-rank Learning for Multivariate Response Regression