High Dimensional Multivariate Regression and Precision Matrix Estimation via Nonconvex Optimization

Abstract

We propose a nonconvex estimator for joint multivariate regression and precision matrix estimation in the high dimensional regime, under sparsity constraints. A gradient descent algorithm with hard thresholding is developed to solve the nonconvex estimator, and it attains a linear rate of convergence to the true regression coefficients and precision matrix simultaneously, up to the statistical error. Compared with existing methods along this line of research, which have little theoretical guarantee, the proposed algorithm not only is computationally much more efficient with provable convergence guarantee, but also attains the optimal finite sample statistical rate up to a logarithmic factor. Thorough experiments on both synthetic and real datasets back up our theory.

32 pages, 5 figures, 3 tables

References in corpus (7)

• Support union recovery in high-dimensional multivariate regression
• A Unified Framework for High-Dimensional Analysis of M-Estimators with Decomposable Regularizers
• Fast low-rank estimation by projected gradient descent: General statistical and algorithmic guarantees
• Regularized M-estimators with nonconvexity: Statistical and algorithmic theory for local optima
• Simple, Efficient, and Neural Algorithms for Sparse Coding
• Gradient Hard Thresholding Pursuit for Sparsity-Constrained Optimization
• Joint estimation of sparse multivariate regression and conditional graphical models