Empirical Bayes PCA in high dimensions
arXiv:2012.11676
Abstract
When the dimension of data is comparable to or larger than the number of data samples, Principal Components Analysis (PCA) may exhibit problematic high-dimensional noise. In this work, we propose an Empirical Bayes PCA method that reduces this noise by estimating a joint prior distribution for the principal components. EB-PCA is based on the classical Kiefer-Wolfowitz nonparametric MLE for empirical Bayes estimation, distributional results derived from random matrix theory for the sample PCs, and iterative refinement using an Approximate Message Passing (AMP) algorithm. In theoretical "spiked" models, EB-PCA achieves Bayes-optimal estimation accuracy in the same settings as an oracle Bayes AMP procedure that knows the true priors. Empirically, EB-PCA significantly improves over PCA when there is strong prior structure, both in simulation and on quantitative benchmarks constructed from the 1000 Genomes Project and the International HapMap Project. An illustration is presented for analysis of gene expression data obtained by single-cell RNA-seq.
References in corpus (22)
- Message Passing Algorithms for Compressed Sensing
- Expectation-Maximization Gaussian-Mixture Approximate Message Passing
- Matrix estimation by Universal Singular Value Thresholding
- Nonlinear shrinkage estimation of large-dimensional covariance matrices
- Sparse principal component analysis and iterative thresholding
- PCA consistency in high dimension, low sample size context
- Sparse PCA: Optimal rates and adaptive estimation
- Universality in polytope phase transitions and message passing algorithms
- General maximum likelihood empirical Bayes estimation of normal means
- OptShrink: An algorithm for improved low-rank signal matrix denoising by optimal, data-driven singular value shrinkage
- Limit theorems for sample eigenvalues in a generalized spiked population model
- A direct formulation for sparse PCA using semidefinite programming
- Mutual information for symmetric rank-one matrix estimation: A proof of the replica formula
- Phase transitions and sample complexity in Bayes-optimal matrix factorization
- MMSE of probabilistic low-rank matrix estimation: Universality with respect to the output channel
- The outliers of a deformed Wigner matrix
- Fundamental limits of low-rank matrix estimation: the non-symmetric case
- Empirical Bayes Matrix Factorization
- An Instability in Variational Inference for Topic Models
- Self-regularizing Property of Nonparametric Maximum Likelihood Estimator in Mixture Models
- Singular vector and singular subspace distribution for the matrix denoising model
- Approximate nonparametric maximum likelihood inference for mixture models via convex optimization