The asymptotic distribution and Berry--Esseen bound of a new test for independence in high dimension with an application to stochastic optimization

Abstract

Let ${\mathbf{X}}_1,...,{\mathbf{X}}_n$ be a random sample from a $p$-dimensional population distribution. Assume that $c_1{n}^{\alpha }\le p\le c_2{n}^{\alpha }$ for some positive constants $c_1,c_2$ and $\alpha$. In this paper we introduce a new statistic for testing independence of the $p$-variates of the population and prove that the limiting distribution is the extreme distribution of type I with a rate of convergence $O((\log n{)}^{5/2}/\sqrt{n})$. This is much faster than $O(1/\log n)$, a typical convergence rate for this type of extreme distribution. A simulation study and application to stochastic optimization are discussed.

arXiv commentPublished in at http://dx.doi.org/10.1214/08-AAP527 the Annals of Applied Probability (http://www.imstat.org/aap/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (2)

• Statistical Challenges with High Dimensionality: Feature Selection in Knowledge Discovery
• The asymptotic distributions of the largest entries of sample correlation matrices

Cited by in corpus (21)

• Distributions of Angles in Random Packing on Spheres
• Test for bandedness of high-dimensional covariance matrices and bandwidth estimation
• Comparing large covariance matrices under weak conditions on the dependence structure and its application to gene clustering
• Necessary and sufficient conditions for the asymptotic distributions of coherence of ultra-high dimensional random matrices
• A test of weak separability for multi-way functional data, with application to brain connectivity studies
• A Cramér moderate deviation theorem for Hotelling's $T^2$-statistic with applications to global tests
• Gaussian Approximation for High Dimensional Time Series
• Asymptotic Inference of Autocovariances of Stationary Processes
• Joint limiting laws for high-dimensional independence tests
• On the theoretic and practical merits of the banding estimator for large covariance matrices
• Central Limit Theorems for Classical Likelihood Ratio Tests for High-Dimensional Normal Distributions
• Asymptotically Independent U-Statistics in High-Dimensional Testing
• Limiting behavior of largest entry of random tensor constructed by high-dimensional data
• On Jiang's asymptotic distribution of the largest entry of a sample correlation matrix
• Large Scale Correlation Screening
• Phase Transition in Limiting Distributions of Coherence of High-Dimensional Random Matrices
• Testing independence with high-dimensional correlated samples
• Coherence of high-dimensional random matrices in a Gaussian case : application of the Chen-Stein method
• Non-uniform Berry-Esseen Bound by Unbounded Exchangeable Pair Approach
• Asymptotic power of Rao's score test for independence in high dimensions
• Limiting Laws of Coherence of Random Matrices with Applications to Testing Covariance Structure and Construction of Compressed Sensing Matrices