Cleaning large-dimensional covariance matrices for correlated samples
arXiv:2107.01352 · doi:10.1103/PhysRevE.105.034136
Abstract
We elucidate the problem of estimating large-dimensional covariance matrices in the presence of correlations between samples. To this end, we generalize the Marcenko-Pastur equation and the Ledoit-Peche shrinkage estimator using methods of random matrix theory and free probability. We develop an efficient algorithm that implements the corresponding analytic formulas, based on the Ledoit-Wolf kernel estimation technique. We also provide an associated open-source Python library, called "shrinkage", with a user-friendly API to assist in practical tasks of estimation of large covariance matrices. We present an example of its usage for synthetic data generated according to exponentially-decaying auto-correlations.
arXiv comment16 pages, 12 figures
References in corpus (7)
- Nonlinear shrinkage estimation of large-dimensional covariance matrices
- Cleaning large correlation matrices: tools from random matrix theory
- Nonhermitean Random Matrix Models : a Free Random Variable Approach
- Lattice QCD determination of neutron-antineutron matrix elements with physical quark masses
- Spectral Moments of Correlated Wishart Matrices
- A Random Matrix Approach to VARMA Processes
- Free products of large random matrices - a short review of recent developments