Shrinkage Algorithms for MMSE Covariance Estimation

Abstract

We address covariance estimation in the sense of minimum mean-squared error (MMSE) for Gaussian samples. Specifically, we consider shrinkage methods which are suitable for high dimensional problems with a small number of samples (large p small n). First, we improve on the Ledoit-Wolf (LW) method by conditioning on a sufficient statistic. By the Rao-Blackwell theorem, this yields a new estimator called RBLW, whose mean-squared error dominates that of LW for Gaussian variables. Second, to further reduce the estimation error, we propose an iterative approach which approximates the clairvoyant shrinkage estimator. Convergence of this iterative method is established and a closed form expression for the limit is determined, which is referred to as the oracle approximating shrinkage (OAS) estimator. Both RBLW and OAS estimators have simple expressions and are easily implemented. Although the two methods are developed from different persepctives, their structure is identical up to specified constants. The RBLW estimator provably dominates the LW method. Numerical simulations demonstrate that the OAS approach can perform even better than RBLW, especially when n is much less than p. We also demonstrate the performance of these techniques in the context of adaptive beamforming.

References in corpus (1)

• Regularized estimation of large covariance matrices

Cited by in corpus (54)

• Robust Shrinkage Estimation of High-dimensional Covariance Matrices
• MOABB: Trustworthy algorithm benchmarking for BCIs
• Adaptive Evolutionary Clustering
• A Geometric Approach to Covariance Matrix Estimation and its Applications to Radar Problems
• Deep CNNs Meet Global Covariance Pooling: Better Representation and Generalization
• Subspace Leakage Analysis and Improved DOA Estimation with Small Sample Size
• Graph Autoencoders for Embedding Learning in Brain Networks and Major Depressive Disorder Identification
• Recognizing Local and Global Structural Motifs at the Atomic Scale
• Optimal shrinkage covariance matrix estimation under random sampling from elliptical distributions
• MIG Median Detectors with Manifold Filter
• Federated Learning via Posterior Averaging: A New Perspective and Practical Algorithms
• Linear Shrinkage Estimation of Covariance Matrices Using Low-Complexity Cross-Validation
• Outage and Finite-SNR DMT Analysis for IRS-aided MIMO Systems: How Large IRSs Need to Be?
• Bayesian Estimation of Graph Signals
• Two-Stage Subspace Constrained Precoding in Massive MIMO Cellular Systems
• REXPACO: an algorithm for high contrast reconstruction of the circumstellar environment by angular differential imaging
• Regularized estimation of linear functionals of precision matrices for high-dimensional time series
• Introduction to correlation networks: Interdisciplinary approaches beyond thresholding
• KaRMMa -- Kappa Reconstruction for Mass Mapping
• Deep Efficient Continuous Manifold Learning for Time Series Modeling
• Regularized Covariance Estimation for Polarization Radar Detection in Compound Gaussian Sea Clutter
• Covariance Matrix Estimation with Non Uniform and Data Dependent Missing Observations
• Widely-Linear MMSE Estimation of Complex-Valued Graph Signals
• Lifelong Machine Learning with Deep Streaming Linear Discriminant Analysis
• Coupled regularized sample covariance matrix estimator for multiple classes
• Newer is not always better: Rethinking transferability metrics, their peculiarities, stability and performance
• Spatial Filtering Pipeline Evaluation of Cortically Coupled Computer Vision System for Rapid Serial Visual Presentation
• Optimal Design of the Adaptive Normalized Matched Filter Detector
• Radiative Transitions in Charmonium from Lattice QCD
• Random Matrix Derived Shrinkage of Spectral Precision Matrices
• Covariance Matrix Estimation from Linearly-Correlated Gaussian Samples
• Compressing and forecasting atomic material simulations with descriptors
• Convex programming approach to robust estimation of a multivariate Gaussian model
• Ensemble Variational Fokker-Planck Methods for Data Assimilation
• Truncated Gaussian-Mixture Variational AutoEncoder
• Post-selection inference with HSIC-Lasso
• Ledoit-Wolf linear shrinkage with unknown mean
• Affine-Invariant Integrated Rank-Weighted Depth: Definition, Properties and Finite Sample Analysis
• On the variability of the sample covariance matrix under complex elliptical distributions
• Regularized Block Toeplitz Covariance Matrix Estimation via Kronecker Product Expansions
• Cross-Validated Tuning of Shrinkage Factors for MVDR Beamforming Based on Regularized Covariance Matrix Estimation
• Linear pooling of sample covariance matrices
• Robust Covariance Matrix Estimation for Radar Space-Time Adaptive Processing (STAP)
• Internal-Coordinate Density Modelling of Protein Structure: Covariance Matters
• Hybrid Projection Methods for Large-scale Inverse Problems with Mixed Gaussian Priors
• Efficient Covariance Estimation from Temporal Data
• Statistical Analysis of the Phosphate Data of the World Ocean Database 2013
• High-Dimensional Covariance Shrinkage for Signal Detection
• Adaptive Shrinkage Estimation for Streaming Graphs
• Ensemble Kalman Filter Implementations Based on Covariance Matrix Estimation
• Graph-LDA: Graph Structure Priors to Improve the Accuracy in Few-Shot Classification
• Detection of Abrupt Change in Channel Covariance Matrix for Multi-Antenna Communication
• Numerical Implementation of the QuEST Function
• Source Enumeration via RMT Estimator Based on Linear Shrinkage Estimation of Noise Eigenvalues Using Relatively Few Samples