High-dimensionality effects in the Markowitz problem and other quadratic programs with linear constraints: Risk underestimation

Abstract

We first study the properties of solutions of quadratic programs with linear equality constraints whose parameters are estimated from data in the high-dimensional setting where p, the number of variables in the problem, is of the same order of magnitude as n, the number of observations used to estimate the parameters. The Markowitz problem in Finance is a subcase of our study. Assuming normality and independence of the observations we relate the efficient frontier computed empirically to the "true" efficient frontier. Our computations show that there is a separation of the errors induced by estimating the mean of the observations and estimating the covariance matrix. In particular, the price paid for estimating the covariance matrix is an underestimation of the variance by a factor roughly equal to 1-p/n. Therefore the risk of the optimal population solution is underestimated when we estimate it by solving a similar quadratic program with estimated parameters. We also characterize the statistical behavior of linear functionals of the empirical optimal vector and show that they are biased estimators of the corresponding population quantities.

Published in at http://dx.doi.org/10.1214/10-AOS795 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (10)

• Regularized estimation of large covariance matrices
• Covariance regularization by thresholding
• Sparse permutation invariant covariance estimation
• Operator norm consistent estimation of large-dimensional sparse covariance matrices
• Tracy--Widom limit for the largest eigenvalue of a large class of complex sample covariance matrices
• Concentration of measure and spectra of random matrices: Applications to correlation matrices, elliptical distributions and beyond
• Noisy Covariance Matrices and Portfolio Optimization II
• Noisy Covariance Matrices and Portfolio Optimization
• Random matrix theory and robust covariance matrix estimation for financial data
• The Student ensemble of correlation matrices: eigenvalue spectrum and Kullback-Leibler entropy

Cited by in corpus (22)

• Cleaning large correlation matrices: tools from random matrix theory
• Estimation of the Global Minimum Variance Portfolio in High Dimensions
• Optimal shrinkage-based portfolio selection in high dimensions
• A Robust Statistics Approach to Minimum Variance Portfolio Optimization
• Regularized estimation of linear functionals of precision matrices for high-dimensional time series
• A nested factor model for non-linear dependences in stock returns
• Geometric sensitivity of random matrix results: consequences for shrinkage estimators of covariance and related statistical methods
• Non-parametric shrinkage mean estimation for quadratic loss functions with unknown covariance matrices
• Preconditioning to comply with the Irrepresentable Condition
• On the dimension effect of regularized linear discriminant analysis
• Which bridge estimator is optimal for variable selection?
• Can we trust the bootstrap in high-dimension?
• Building portfolios of stocks in the São Paulo Stock Exchange using Random Matrix Theory
• Certifiably Optimal Sparse Inverse Covariance Estimation
• An Overview on the Estimation of Large Covariance and Precision Matrices
• Optimal Covariance Estimation for Condition Number Loss in the Spiked Model
• Is the empirical out-of-sample variance an informative risk measure for the high-dimensional portfolios?
• Optimal estimation of functionals of high-dimensional mean and covariance matrix
• Sparse Portfolio selection via Bayesian Multiple testing
• Network induced large correlation matrix estimation
• Asymptotics For High Dimensional Regression M-Estimates: Fixed Design Results
• High-dimensional covariance matrix estimation using a low-rank and diagonal decomposition