Detection limits in the high-dimensional spiked rectangular model

Abstract

We study the problem of detecting the presence of a single unknown spike in a rectangular data matrix, in a high-dimensional regime where the spike has fixed strength and the aspect ratio of the matrix converges to a finite limit. This setup includes Johnstone's spiked covariance model. We analyze the likelihood ratio of the spiked model against an "all noise" null model of reference, and show it has asymptotically Gaussian fluctuations in a region below---but in general not up to---the so-called BBP threshold from random matrix theory. Our result parallels earlier findings of Onatski et al.\ (2013) and Johnstone-Onatski (2015) for spherical spikes. We present a probabilistic approach capable of treating generic product priors. In particular, sparsity in the spike is allowed. Our approach is based on Talagrand's interpretation of the cavity method from spin-glass theory. The question of the maximal parameter region where asymptotic normality is expected to hold is left open. This region is shaped by the prior in a non-trivial way. We conjecture that this is the entire paramagnetic phase of an associated spin-glass model, and is defined by the vanishing of the replica-symmetric solution of Lesieur et al.\ (2015).

arXiv comment28 pages. Appears in the proc. of the 31st annual Conference on Learning Theory (COLT) 2018

References in corpus (14)

• Optimal detection of sparse principal components in high dimension
• The largest eigenvalue of rank one deformation of large Wigner matrices
• Limit theorems for sample eigenvalues in a generalized spiked population model
• Mutual information for symmetric rank-one matrix estimation: A proof of the replica formula
• MMSE of probabilistic low-rank matrix estimation: Universality with respect to the output channel
• The free energy in a multi-species Sherrington-Kirkpatrick model
• Equilibrium statistical mechanics of bipartite spin systems
• Mutual Information in Rank-One Matrix Estimation
• A note on classical ground state energies
• Free energy and complexity of spherical bipartite models
• Fluctuations of the free energy of the spherical Sherrington-Kirkpatrick model with ferromagnetic interaction
• Fundamental limits of low-rank matrix estimation: the non-symmetric case
• Sharp detection in PCA under correlations: all eigenvalues matter
• Finite Size Corrections and Likelihood Ratio Fluctuations in the Spiked Wigner Model

Cited by in corpus (6)

• Fundamental limits of detection in the spiked Wigner model
• All-or-nothing statistical and computational phase transitions in sparse spiked matrix estimation
• Phase transition in random tensors with multiple independent spikes
• Contextual Stochastic Block Model: Sharp Thresholds and Contiguity
• Asymptotic normality and analysis of variance of log-likelihood ratios in spiked random matrix models
• Efficient inference in stochastic block models with vertex labels