Noise Dressing of Financial Correlation Matrices

Abstract

We show that results from the theory of random matrices are potentially of great interest to understand the statistical structure of the empirical correlation matrices appearing in the study of price fluctuations. The central result of the present study is the remarkable agreement between the theoretical prediction (based on the assumption that the correlation matrix is random) and empirical data concerning the density of eigenvalues associated to the time series of the different stocks of the S&P500 (or other major markets). In particular the present study raises serious doubts on the blind use of empirical correlation matrices for risk management.

Latex (Revtex) 3 pp + 2 postscript figures (in-text)

References in corpus (4)

• Rational Decisions, Random Matrices and Spin Glasses
• Universality in Chiral Random Matrix Theory at $\beta =1$ and $\beta =4$
• Renormalizing Rectangles and Other Topics in Random Matrix Theory
• Random matrix ensembles with an effective extensive external charge

Cited by in corpus (365)

• Detrended Cross-Correlation Analysis: A New Method for Analyzing Two Non-stationary Time Series
• Universal and non-universal properties of cross-correlations in financial time series
• A Random Matrix Approach to Cross-Correlations in Financial Data
• Dynamics of market correlations: Taxonomy and portfolio analysis
• Scaling of the distribution of price fluctuations of individual companies
• Spectra of "Real-World" Graphs: Beyond the Semi-Circle Law
• Hyperuniform States of Matter
• Topology of correlation based minimal spanning trees in real and model markets
• Statistical physics of inference: Thresholds and algorithms
• Induced measures in the space of mixed quantum states
• Networks of equities in financial markets
• Multifractal analysis of financial markets
• Correlation, hierarchies, and networks in financial markets
• Dynamic asset trees and Black Monday
• Correlation of financial markets in times of crisis
• The Physics of Financial Networks
• Weighted network modules
• Multiscale characteristics of the emerging global cryptocurrency market
• Spectra and eigenvectors of scale-free networks
• Evolution of worldwide stock markets, correlation structure and correlation based graphs
• Cleaning large correlation matrices: tools from random matrix theory
• Correlation based networks of equity returns sampled at different time horizons
• Random Matrix Theory Analysis of Cross Correlations in Financial Markets
• Volatility clustering and scaling for financial time series due to attractor bubbling
• Taxonomy of Stock Market Indices
• Collective behavior of stock price movements in an emerging market
• Correlation, Network and Multifractal Analysis of Global Financial Indices
• Mutual Information Rate-Based Networks in Financial Markets
• Testing statistical bounds on entanglement using quantum chaos
• Worldwide spreading of economic crisis
• Systematic analysis of group identification in stock markets
• Spanning Trees and bootstrap reliability estimation in correlation based networks
• Community detection for correlation matrices
• Topology of Foreign Exchange Markets using Hierarchical Structure Methods
• Quantifying and Modeling Long-Range Cross-Correlations in Multiple Time Series with Applications to World Stock Indices
• Spectra of Sparse Random Matrices
• Cross-correlation in financial dynamics
• Dynamics of competition between collectivity and noise in the stock market
• Temporal Evolution of Financial Market Correlations
• Characteristics of the Korean stock market correlations
• Sector analysis for a FTSE portfolio of stocks
• Universal and nonuniversal allometric scaling behaviors in the visibility graphs of world stock market indices
• Approximate amplitude encoding in shallow parameterized quantum circuits and its application to financial market indicator
• Scale-free Network in Financial Correlations
• Systemic risk and spatiotemporal dynamics of the US housing market
• A model for correlations in stock markets
• Statistics of Atmospheric Correlations
• Noisy Covariance Matrices and Portfolio Optimization II
• Correlations in commodity markets
• Levels of complexity in financial markets
• Variety and Volatility in Financial Markets
• Estimated Correlation Matrices and Portfolio Optimization
• Stock price jumps: news and volume play a minor role
• Emergence of time-horizon invariant correlation structure in financial returns by subtraction of the market mode
• Random matrix analysis of complex networks
• Structure of local interactions in complex financial dynamics
• A New Method to Estimate the Noise in Financial Correlation Matrices
• Towards identifying the world stock market cross-correlations: DAX versus Dow Jones
• On the localization transition in symmetric random matrices
• Increasing market efficiency: Evolution of cross-correlations of stock returns
• An analysis of Cross-correlations in South African Market data
• Cross-Correlation Dynamics in Financial Time Series
• Market dynamics immediately before and after financial shocks: quantifying the Omori, productivity and Bath laws
• Data clustering and noise undressing for correlation matrices
• Kullback-Leibler distance as a measure of the information filtered from multivariate data
• The structural role of weak and strong links in a financial market network
• Evolution of correlation structure of industrial indices of US equity markets
• On the largest eigenvalue of Wishart matrices with identity covariance when n, p and p/n tend to infinity
• The bulk of the stock market correlation matrix is not pure noise
• Dissecting cross-impact on stock markets: An empirical analysis
• Will the US Economy Recover in 2010? A Minimal Spanning Tree Study
• Specialization of strategies and herding behavior of trading firms in a financial market
• Evaluating the RiskMetrics Methodology in Measuring Volatility and Value-at-Risk in Financial Markets
• Random Matrix Theory and Fund of Funds Portfolio Optimisation
• Identifying phase synchronization clusters in spatially extended dynamical systems
• Gaussian fluctuations for linear spectral statistics of large random covariance matrices
• Spectral Moments of Correlated Wishart Matrices
• Level statistics and localization transitions of Lévy matrices
• Performance analysis and optimal selection of large mean-variance portfolios under estimation risk
• Collective Origin of the Coexistence of Apparent RMT Noise and Factors in Large Sample Correlation Matrices
• Non-Stationarity in Financial Time Series and Generic Features
• A Robust Statistics Approach to Minimum Variance Portfolio Optimization
• Hierarchically nested factor model from multivariate data
• Dynamic evolution of cross-correlations in the Chinese stock market
• Analysis of cross-correlations in electroencephalogram signals as an approach to proactive diagnosis of schizophrenia
• Large Scale Cross-Correlations in Internet Traffic
• Trading strategies in the Italian interbank market
• Adaptive financial networks with static and dynamic thresholds
• Anti-correlation and subsector structure in financial systems
• Signal and Noise in Financial Correlation Matrices
• Elementary Excitation Modes in a Granular Glass above Jamming
• Portfolio Optimization and the Random Magnet Problem
• Subsystem dynamics under random Hamiltonian evolution
• Competition of noise and collectivity in global cryptocurrency trading: route to a self-contained market
• Econophysics: Empirical facts and agent-based models
• Hausdorff clustering of financial time series
• Carbon-dioxide emissions trading and hierarchical structure in worldwide finance and commodities markets
• Collective correlations, dynamics, and behavioural inconsistencies of the cryptocurrency market over time
• Spectral and network methods in the analysis of correlation matrices of stock returns
• Correlation structure and principal components in global crude oil market
• Uncovering the Dynamics of Correlation Structures Relative to the Collective Market Motion
• Measures of globalization based on cross-correlations of world financial indices
• Topological structures in the equities market network
• Quantitative law describing market dynamics before and after interest-rate change
• Mesoscopic Community Structure of Financial Markets Revealed by Price and Sign Fluctuations
• Singular value distribution of the propagation matrix in random scattering media
• Statistical Properties of Cross-Correlation in the Korean Stock Market
• Random Matrix Theory and Cross-correlations in Global Financial Indices and Local Stock Market Indices
• Identifying long-term precursors of financial market crashes using correlation patterns
• Scale free effects in world currency exchange network
• Traditional and Heavy-Tailed Self Regularization in Neural Network Models
• On the Eigenvalue Density of Real and Complex Wishart Correlation Matrices
• Market correlation structure changes around the Great Crash
• Dynamics, behaviours, and anomaly persistence in cryptocurrencies and equities surrounding COVID-19
• Time scales involved in market emergence
• Cause and Cure of Sloppiness in Ordinary Differential Equation Models
• Agent-based model with multi-level herding for complex financial systems
• Quantifying dynamics of the financial correlations
• Crashes, Recoveries, and `Core-shifts' in a Model of Evolving Networks
• Elements of Coevolution in Biological Sequences
• The q-dependent detrended cross-correlation analysis of stock market
• Spatial Distribution of Phase Singularities in Optical Random Vector Waves
• Dynamic instability in a phenomenological model of correlated assets
• Moments of Wishart-Laguerre and Jacobi ensembles of random matrices: application to the quantum transport problem in chaotic cavities
• Linear and nonlinear market correlations: characterizing financial crises and portfolio optimization
• Configuration model for correlation matrices preserving the node strength
• World currency exchange rate cross-correlations
• Maximal spanning trees, asset graphs and random matrix denoising in the analysis of dynamics of financial networks
• Characterizing Multi-Scale Self-Similar Behavior and Non-Statistical Properties of Financial Time Series
• Random Matrix Spectra as a Time Series
• Random Matrix approach to collective behavior and bulk universality in protein dynamics
• Economic state classification and portfolio optimisation with application to stagflationary environments
• Modelling gap-size distribution of parked cars using random-matrix theory
• Persistence Probabilities of the German DAX and Shanghai Index
• How much random a random network is : a random matrix analysis
• Risk Minimization through Portfolio Replication
• First eigenvalue/eigenvector in sparse random symmetric matrices: influences of degree fluctuation
• On return-volatility correlation in financial dynamics
• Power-law deformation of Wishart-Laguerre ensembles of random matrices
• The fine structure of spectral properties for random correlation matrices: an application to financial markets
• Spectral Density of Sparse Sample Covariance Matrices
• Spectral properties of empirical covariance matrices for data with power-law tails
• On the concentration of large deviations for fat tailed distributions, with application to financial data
• Regularizing Portfolio Optimization
• Forecasting market states
• Spectral analysis of deformed random networks
• A new measure between sets of probability distributions with applications to erratic financial behavior
• Influence network in Chinese stock market
• Correlations and Flow of Information between The New York Times and Stock Markets
• Universal Correlations and Power-Law Tails in Financial Covariance Matrices
• Distribution of the Smallest Eigenvalue in the Correlated Wishart Model
• Credit risk - A structural model with jumps and correlations
• Club Convergence of House Prices: Evidence from China's Ten Key Cities
• U.S. stock market interaction network as learned by the Boltzmann Machine
• Collective dynamics, diversification and optimal portfolio construction for cryptocurrencies
• Cross correlations of the American baby names
• Introduction to correlation networks: Interdisciplinary approaches beyond thresholding
• Supersymmetry approach to Wishart correlation matrices: Exact results
• Large complex correlated Wishart matrices: Fluctuations and asymptotic independence at the edges
• Forecasting the underlying potential governing the time series of a dynamical system
• The effect of a market factor on information flow between stocks using minimal spanning tree
• A Random Matrix Approach to VARMA Processes
• A nested factor model for non-linear dependences in stock returns
• Distribution of the ratio of two consecutive level spacings in orthogonal to unitary crossover ensembles
• A Study of Correlations in the Stock Market
• Determining the number of factors in a forecast model by a random matrix test: cryptocurrencies
• Symmetry deduction from spectral fluctuations in complex quantum systems
• Eigenvalue statistics for the sum of two complex Wishart matrices
• On the non-stationarity of financial time series: impact on optimal portfolio selection
• Scaling in the eigenvalue fluctuations of the empirical correlation matrices
• Asymmetric correlation matrices: an analysis of financial data
• Family of solvable generalized random-matrix ensembles with unitary symmetry
• Replica approach to mean-variance portfolio optimization
• Hierarchical structure of the European countries based on debts as a percentage of GDP during the 2000-2011 period
• Zooming into market states
• Multiscaled Cross-Correlation Dynamics in Financial Time-Series
• Constraining Primordial Non-Gaussianities from the WMAP2 2-1 Cumulant Correlator Power Spectrum
• Dependency structures in cryptocurrency market from high to low frequency
• Spectral analysis of finite-time correlation matrices near equilibrium phase transitions
• Statistical pairwise interaction model of stock market
• Cross-correlation asymmetries and causal relationships between stock and market risk
• State and group dynamics of world stock market by principal component analysis
• Spectra of large time-lagged correlation matrices from Random Matrix Theory
• Random Matrix Theory and the Failure of Macroeconomic Forecasts
• Constructing networks by filtering correlation matrices: A null model approach
• Non-Hermitean Wishart random matrices (I)
• A New Attempt to Identify Long-term Precursors for Endogenous Financial Crises in the Market Correlation Structures
• Estimating the principal components of correlation matrices from all their empirical eigenvectors
• Identifying Complexity by Means of Matrices
• Uncovering functional brain signature via random matrix theory
• Emerging spectra of singular correlation matrices under small power-map deformations
• Evolutionary correlation, regime switching, spectral dynamics and optimal trading strategies for cryptocurrencies and equities
• Instabilities of complex fluids with partially structured and partially random interactions
• Semi-metric portfolio optimization: a new algorithm reducing simultaneous asset shocks
• The rank 1 real Wishart spiked model
• Universal Spectral Correlations in Orthogonal-Unitary and Symplectic-Unitary Crossover Ensembles of Random Matrices
• Spectral methods and cluster structure in correlation-based networks
• Spectral correlation functions of the sum of two independent complex Wishart matrices with unequal covariances
• Spectral Domain of Large Nonsymmetric Correlated Wishart Matrices
• A statistical physics perspective on criticality in financial markets
• Random matrix analysis of deep neural network weight matrices
• Random matrix approach to the dynamics of stock inventory variations
• Correlated Wishart Matrices and Critical Horizons
• The Convergence of European Business Cycles 1978-2000
• Emergence of frustration signals systemic risk
• Dynamics of Bid-ask Spread Return and Volatility of the Chinese Stock Market
• Hausdorff clustering
• Alternation of different fluctuation regimes in the stock market dynamics
• Comparison of the superbosonization formula and the generalized Hubbard-Stratonovich transformation
• Nature of order from random two-body interactions
• How to quantify the influence of correlations on investment diversification
• Norm-dependent Random Matrix Ensembles in External Field and Supersymmetry
• Financial time-series analysis: A brief overview
• Random matrix approach to multivariate categorical data analysis
• Correlations and Clustering in Wholesale Electricity Markets
• Memory effect and multifractality of cross-correlations in financial markets
• An exploration of the mathematical structure and behavioural biases of 21st century financial crises
• A dynamical model for competing opinions
• A Structural Model for Fluctuations in Financial Markets
• Random, but not so much: A parameterization for the returns and correlation matrix of financial time series
• Spectral density of a Wishart model for nonsymmetric Correlation Matrices
• Spatial and temporal structures of four financial markets in Greater China
• Correlation structure analysis of the global agricultural futures market
• Spectrum, Intensity and Coherence in Weighted Networks of a Financial Market
• Non-Stationary Covariance Matrices And Noise
• Scaling Features of Price-Volume Cross-Correlation
• A Survey on the Eigenvalues Local Behavior of Large Complex Correlated Wishart Matrices
• Signal detection in nearly continuous spectra and symmetry breaking
• Maximum Entropy approach to multivariate time series randomization
• Quantifying the status of economies in international crop trade networks: An correlation structure analysis of various node-ranking metrics
• New Collectivity Measures for Financial Covariances and Correlations
• Building portfolios of stocks in the São Paulo Stock Exchange using Random Matrix Theory
• Spectral densities of Wishart-Levy free stable random matrices: Analytical results and Monte Carlo validation
• Distribution of the Smallest Eigenvalue in Complex and Real Correlated Wishart Ensembles
• Collective behavior in the North Rhine-Westphalia motorway network
• Supersymmetry Method for Chiral Random Matrix Theory with Arbitrary Rotation Invariant Weights
• Limiting Statistics of the Largest and Smallest Eigenvalues in the Correlated Wishart Model
• Correlation Patterns in Foreign Exchange Markets
• Complementarity and Related Problems
• Projecting XRP price burst by correlation tensor spectra of transaction networks
• Nonparametric sign prediction of high-dimensional correlation matrix coefficients
• Eigenfunction statistics of Laguerre Brownian ensemble
• Effect of changing data size on eigenvalues in the Korean and Japanese stock markets
• Decomposing the stock market intraday dynamics
• Saturation of number variance in embedded random matrix ensembles
• Optimization of Trading Physics Models of Markets
• Correlations and clustering in the trading of members of the London Stock Exchange
• From human mobility to renewable energies: Big data analysis to approach worldwide multiscale phenomena
• Constructing Analytically Tractable Ensembles of Non-Stationary Covariances with an Application to Financial Data
• Finite-Range Coulomb Gas Models of Banded Random Matrices and Quantum Kicked Rotors
• Localization in random bipartite graphs: numerical and empirical study
• Transitions in the Stock Markets of the US, UK, and Germany
• Correlations versus noise in the NFT market
• A Random Matrix Approach to Dynamic Factors in macroeconomic data
• Memory Effects, Multiple Time Scales and Local Stability in Langevin Models of the S&P500 Market Correlation
• Phase Transition in the S&P Stock Market
• Finite-Range Coulomb Gas Models I: Some Analytical Results
• Uncovering the mesoscale structure of the credit default swap market to improve portfolio risk modelling
• Deep Reinforcement Learning for Long-Short Portfolio Optimization
• A new spin on optimal portfolios and ecological equilibria
• On the systemic nature of global inflation, its association with equity markets and financial portfolio implications
• Uncovering the Internal Structure of the Indian Financial Market: Cross-correlation behavior in the NSE
• Serial Correlation, Periodicity and Scaling of Eigenmodes in an Emerging Market
• Correlation Matrix Spectra: A Tool for Detecting Non-apparent Correlations?
• Binary versus non-binary information in real time series: empirical results and maximum-entropy matrix models
• Free Random Levy Variables and Financial Probabilities
• Eigenvector localization as a tool to study small communities in online social networks
• Application of spectral methods for high-frequency financial data to quantifying states of market participants
• Fluctuations in Mass-Action Equilibrium of Protein Binding Networks
• Identifying Dominant Industrial Sectors in Market States of the S&P 500 Financial Data
• Implied Basket Correlation Dynamics
• Localization in covariance matrices of coupled heterogenous Ornstein-Uhlenbeck processes
• Analysis of delay correlation matrices
• Spectra of Empirical Auto-Covariance Matrices
• Two-step estimators of high dimensional correlation matrices
• Eigenvalue and Eigenvector Statistics in Time Series Analysis
• Portfolio Optimization on Multivariate Regime Switching GARCH Model with Normal Tempered Stable Innovation
• Power law eigenvalue density, scaling and critical random matrix ensembles
• Predicting trend reversals using market instantaneous state
• Complex Correlation Approach for High Frequency Financial Data
• Note on log-periodic description of 2008 financial crash
• Optimal design of experiments by combining coarse and fine measurements
• Statistical Analysis on Random Quantum Sampling by Sycamore and Zuchongzhi Quantum Processors
• Learning Clique Forests
• Universality of a family of Random Matrix Ensembles with logarithmic soft-confinement potentials
• Filtering of complex systems using overlapping tree networks
• Exact finite-size scaling for the random-matrix representation of bond percolation on square lattice
• Unveiling the directional network behind the financial statements data using volatility constraint correlation
• Optimal Allocation of Trend Following Strategies
• Coherence and incoherence collective behavior in financial market
• Collective infectivity of the pandemic over time and association with vaccine coverage and economic development
• Random matrix theory and nested clustered portfolios on Mexican markets
• Efficient Multi-Change Point Analysis to decode Economic Crisis Information from the S&P500 Mean Market Correlation
• Accounting for risk of non linear portfolios: a novel Fourier approach
• Spectral density of the non-central correlated Wishart ensembles
• Rotationally invariant family of Lévy like random matrix ensembles
• Random matrix approach for primal-dual portfolio optimization problems
• Matrix H-theory approach to stock market fluctuations
• Exact Multivariate Amplitude Distributions for Non-Stationary Gaussian or Algebraic Fluctuations of Covariances or Correlations
• A memory-based method to select the number of relevant components in Principal Component Analysis
• Geography and distance effect on financial dynamics in the Chinese stock market
• A 2D Levy-flight model for the complex dynamics of real-life financial markets
• Describing the effect of influential spreaders on the different sectors of Indian market: a complex networks perspective
• Analysis of Inter-Domain Traffic Correlations: Random Matrix Theory Approach
• Universality and scaling in human and social systems
• Grasping asymmetric information in market impacts
• Learning interactions between Rydberg atoms
• Detecting imbalanced financial markets through time-varying optimization and nonlinear functionals
• Quantifying the information lost in optimal covariance matrix cleaning
• Eigenvalue Distribution of Empirical Correlation Matrices for Multiscale Complex Systems and Application to Financial Data
• Interaction-correlated random matrices
• Dynamic structure of stock communities: A comparative study between stock returns and turnover rates
• Cluster formation and evolution in networks of financial market indices
• Mesoscopic Structure of the Stock Market and Portfolio Optimization
• On financial market correlation structures and diversification benefits across and within equity sectors
• Random matrix ensemble for the covariance matrix of Ornstein-Uhlenbeck processes with heterogeneous temperatures
• Multivariate Garch with dynamic beta
• End-to-End Large Portfolio Optimization for Variance Minimization with Neural Networks through Covariance Cleaning
• Boundary between noise and information applied to filtering neural network weight matrices
• Emergence of correlations between securities at short time scales
• Counting Multiple Solutions in Glassy Random Matrix Models
• Assessing the Predictability of Social and Economic Time-Series Data: the example of crime in the UK
• A Bayesian Graphical Approach for Large-Scale Portfolio Management with Fewer Historical Data
• Trading in Risk Dimensions (TRD)
• Experts' earning forecasts: bias, herding and gossamer information
• Large-Scale Simulation of Multi-Asset Ising Financial Markets
• Asymptotic Coincidence of the Statistics for Degenerate and Non-Degenerate Correlated Real Wishart Ensembles
• Statistical properties of power-law random banded unitary matrices in the delocalization-localization transition regime
• Random matrix theory and the evolution of business cycle synchronisation 1886-2006
• Statistical properties of information flow in financial time series
• Signal from noise retrieval from one and two-point Green's function - comparison
• Random selection of factors preserves the correlation structure in a linear factor model to a high degree
• Surprising Pfaffian factorizations in Random Matrix Theory with Dyson index $\beta =2$
• Random matrix approach in search for weak signals immersed in background noise
• Dynamic analysis of influential stocks based on conserved networks
• $L_p$ regularized portfolio optimization
• Generative random latent features models and statistics of natural images
• Portfolio Optimization under Expected Shortfall: Contour Maps of Estimation Error
• Inferring financial stock returns correlation from complex network analysis
• Matrix Moments in a Real, Doubly Correlated Algebraic Generalization of the Wishart Model
• Patterns for the waiting time in the context of discrete-time stochastic processes
• A study of co-movements between USA and Latin American stock markets: a cross-bicorrelations perspective
• Characterizing price index behavior through fluctuation dynamics
• Removing noise from correlations in multivariate stock price data
• The Effects of Market Properties on Portfolio Diversification in the Korean and Japanese Stock Markets
• Dynamics of the market states in the space of correlation matrices with applications to financial markets
• COVID anomaly in the correlation analysis of S&P 500 market states
• A New Traders' Game? -- Empirical Analysis of Response Functions in a Historical Perspective
• Estimating correlation and covariance matrices by weighting of market similarity
• From Space and Time to a Deeper Reality as a Possible Way to Solve Global Problems
• Uncovering networks amongst stocks returns by studying nonlinear interactions in high frequency data of the Indian Stock Market using mutual information
• Plunges in the Bombay stock exchange: Characteristics and indicators
• Multivariate Distributions in Non-Stationary Complex Systems II: Empirical Results for Correlated Stock Markets
• Generic Features in the Spectral Decomposition of Correlation Matrices
• Toy Model for Large Non-Symmetric Random Matrices
• Signal inference in financial stock return correlations through phase-ordering kinetics in the quenched regime
• A FEAST variant incorporated with a power iteration
• Optimal trading strategies - a time series approach
• Assessment of Spatial-variant Noise Level in Real-time Dynamic MR Images - a Random Matrix Approach
• The evolution of EU business cycle synchronisation 1981-2007
• Renormalization group for spectral collapse in random matrices with power-law variance profiles
• Functional Renormalization for Signal Detection: Dimensional Analysis and Dimensional Phase Transition for Nearly Continuous Spectra Effective Field Theory
• On the extremal spectral properties of random graphs
• Trading characteristics of member firms on the Korea Exchange
• Random Matrix Application to Correlations Among Volatility of Assets