Functional Renormalization for Signal Detection: Dimensional Analysis and Dimensional Phase Transition for Nearly Continuous Spectra Effective Field Theory

Abstract

Signal detection in high dimensions is a critical challenge in data science. While standard methods based on random matrix theory provide sharp detection thresholds for finite-rank perturbations, such as the known Baik-Ben Arous-Péché (BBP) transition, they are often insufficient for realistic data exhibiting nearly continuous (extensive-rank) signal distributions that merge with the noise bulk. In this regime, typically associated with real-world scenarios such as images for computer vision tasks, the signal does not manifest as a clear outlier but as a deformation of the spectral density's geometry. We use the functional renormalisation group (FRG) framework to probe these subtle spectral deformations. Treating the empirical spectrum as an effective field theory, we define a scale-dependent "canonical dimension" that acts as a sensitive order parameter for the spectral geometry. We show that this dimension undergoes a sharp crossover, interpreted as a "dimensional phase transition", at signal-to-noise ratios significantly lower than the standard BBP threshold. This dimensional instability is shown to correlate with a spontaneous symmetry breaking in the effective potential and a deviation of eigenvector statistics from the universal Porter-Thomas distribution, confirming the consistency of the method. Such behaviour aligns with recent theoretical results on the "extensive spike model", where signal information persists inside the noise bulk before any spectral gap opens. We validate our approach on realistic datasets, demonstrating that the FRG flow consistently detects the onset of this bulk deformation. Finally, we explore a formalisation of this methodology for analysing nearly continuous spectra, proposing a heuristic criterion for signal detection and a method to estimate the number of independent noise components based on the stability of these canonical dimensions.

36 pages; update figures

References in corpus (23)

• SciPy 1.0--Fundamental Algorithms for Scientific Computing in Python
• Array Programming with NumPy
• Inverse statistical problems: from the inverse Ising problem to data science
• PCA meets RG
• Neural Networks and Quantum Field Theory
• Nonperturbative renormalization group beyond melonic sector: The Effective Vertex Expansion method for group fields theories
• Nonperturbative renormalization for the neural network-QFT correspondence
• The renormalisation group via statistical inference
• Modification of the Porter-Thomas distribution by rank-one interaction
• HyperPCA: a Powerful Tool to Extract Elemental Maps from Noisy Data Obtained in LIBS Mapping of Materials
• Field theoretical approach for signal detection in nearly continuous positive spectra II: Tensorial data
• Field theoretical approach for signal detection in nearly continuous positive spectra I: Matricial data
• Signal detection in nearly continuous spectra and symmetry breaking
• The Inverse of Exact Renormalization Group Flows as Statistical Inference
• Physical model of dimensional regularization
• Singular Vectors of Sums of Rectangular Random Matrices and Optimal Estimators of High-Rank Signals: The Extensive Spike Model
• Inferring relevant features: from QFT to PCA
• Sharp complexity asymptotics and topological trivialization for the (p, k) spiked tensor model
• Functional renormalization group for signal detection and stochastic ergodicity breaking
• An intriguing connection between Pisarski's fixed point and (2+3)-spin glasses
• Functional renormalization group for $p=2$ like glassy matrices in the planar approximation: II. Ward identities method in the deep IR
• Functional Renormalization Group Approach for Signal Detection
• Functional renormalization group for p=2 like glassy matrices in the planar approximation: I. Vertex expansion at equilibrium