A memory-based method to select the number of relevant components in Principal Component Analysis

Abstract

We propose a new data-driven method to select the optimal number of relevant components in Principal Component Analysis (PCA). This new method applies to correlation matrices whose time autocorrelation function decays more slowly than an exponential, giving rise to long memory effects. In comparison with other available methods present in the literature, our procedure does not rely on subjective evaluations and is computationally inexpensive. The underlying basic idea is to use a suitable factor model to analyse the residual memory after sequentially removing more and more components, and stopping the process when the maximum amount of memory has been accounted for by the retained components. We validate our methodology on both synthetic and real financial data, and find in all cases a clear and computationally superior answer entirely compatible with available heuristic criteria, such as cumulative variance and cross-validation.

arXiv comment29 pages, published

References in corpus (17)

• Uncovering the overlapping community structure of complex networks in nature and society
• Benchmark graphs for testing community detection algorithms
• Hierarchical Structure in Financial Markets
• Noise Dressing of Financial Correlation Matrices
• A Random Matrix Approach to Cross-Correlations in Financial Data
• A tool for filtering information in complex systems
• Cleaning large correlation matrices: tools from random matrix theory
• Introduction to Random Matrices - Theory and Practice
• Complex Networks on Hyperbolic Surfaces
• Relation between Financial Market Structure and the Real Economy: Comparison between Clustering Methods
• Emergence of time-horizon invariant correlation structure in financial returns by subtraction of the market mode
• A New Method to Estimate the Noise in Financial Correlation Matrices
• Parsimonious modeling with Information Filtering Networks
• Spectral Moments of Correlated Wishart Matrices
• Exploring complex networks via topological embedding on surfaces
• The fine structure of spectral properties for random correlation matrices: an application to financial markets
• Complex Correlation Approach for High Frequency Financial Data