Performance of Empirical Risk Minimization For Principal Component Regression

Abstract

This paper studies the predictive performance of empirical risk minimization for principal component regression. Our analysis accommodates the leading eigenvalues of the predictor covariance matrix growing either linearly or sublinearly with the number of predictors. Additionally, we allow for both light-tailed and heavy-tailed data. Our main result establishes that, under appropriate conditions, empirical risk minimization for principal component regression is consistent for prediction and achieves near-optimal performance.