Law of Log Determinant of Sample Covariance Matrix and Optimal Estimation of Differential Entropy for High-Dimensional Gaussian Distributions

Abstract

Differential entropy and log determinant of the covariance matrix of a multivariate Gaussian distribution have many applications in coding, communications, signal processing and statistical inference. In this paper we consider in the high dimensional setting optimal estimation of the differential entropy and the log-determinant of the covariance matrix. We first establish a central limit theorem for the log determinant of the sample covariance matrix in the high dimensional setting where the dimension $p(n)$ can grow with the sample size $n$. An estimator of the differential entropy and the log determinant is then considered. Optimal rate of convergence is obtained. It is shown that in the case $p(n)/n\to 0$ the estimator is asymptotically sharp minimax. The ultra-high dimensional setting where $p(n)>n$ is also discussed.

19 pages

References in corpus (4)

• Optimal rates of convergence for covariance matrix estimation
• Optimal rates of convergence for sparse covariance matrix estimation
• Random matrices: Law of the determinant
• Asymptotic behavior of random determinants in the Laguerre, Gram and Jacobi ensembles

Cited by in corpus (10)

• Barlow Twins: Self-Supervised Learning via Redundancy Reduction
• How Well Generative Adversarial Networks Learn Distributions
• Exponential growth of random determinants beyond invariance
• Mean-field theory of Bayesian clustering
• Optimal Uncertainty Size in Distributionally Robust Inverse Covariance Estimation
• Pointly-supervised 3D Scene Parsing with Viewpoint Bottleneck
• Understanding Neural Networks with Logarithm Determinant Entropy Estimator
• Configurable photonic simulator for quantum field dynamics
• Linear spectral statistics of sequential sample covariance matrices
• Bernstein-von Mises Theorems for Functionals of Covariance Matrix