The Hardness of Conditional Independence Testing and the Generalised Covariance Measure

Abstract

It is a common saying that testing for conditional independence, i.e., testing whether whether two random vectors $X$ and $Y$ are independent, given $Z$, is a hard statistical problem if $Z$ is a continuous random variable (or vector). In this paper, we prove that conditional independence is indeed a particularly difficult hypothesis to test for. Valid statistical tests are required to have a size that is smaller than a predefined significance level, and different tests usually have power against a different class of alternatives. We prove that a valid test for conditional independence does not have power against any alternative. Given the non-existence of a uniformly valid conditional independence test, we argue that tests must be designed so their suitability for a particular problem may be judged easily. To address this need, we propose in the case where $X$ and $Y$ are univariate to nonlinearly regress $X$ on $Z$, and $Y$ on $Z$ and then compute a test statistic based on the sample covariance between the residuals, which we call the generalised covariance measure (GCM). We prove that validity of this form of test relies almost entirely on the weak requirement that the regression procedures are able to estimate the conditional means $X$ given $Z$, and $Y$ given $Z$, at a slow rate. We extend the methodology to handle settings where $X$ and $Y$ may be multivariate or even high-dimensional. While our general procedure can be tailored to the setting at hand by combining it with any regression technique, we develop the theoretical guarantees for kernel ridge regression. A simulation study shows that the test based on GCM is competitive with state of the art conditional independence tests. Code is available as the R package GeneralisedCovarianceMeasure on CRAN.

arXiv commentA proof of Lemma 31 has been added. The earlier (and published) version referenced a book for this proof, but it was subsequently discovered that the statement in the book was incorrect

References in corpus (8)

• XGBoost: A Scalable Tree Boosting System
• Kernel-based Conditional Independence Test and Application in Causal Discovery
• Gaussian approximations and multiplier bootstrap for maxima of sums of high-dimensional random vectors
• Asymptotic normality and optimalities in estimation of large Gaussian graphical models
• Higher order influence functions and minimax estimation of nonlinear functionals
• Goodness of fit tests for high-dimensional linear models
• A Scalable Conditional Independence Test for Nonlinear, Non-Gaussian Data
• On the Equivalence between Kernel Quadrature Rules and Random Feature Expansions

Cited by in corpus (15)

• Causality for Machine Learning
• Demystifying statistical learning based on efficient influence functions
• Causality in cognitive neuroscience: concepts, challenges, and distributional robustness
• A Guide to Feature Importance Methods for Scientific Inference
• Asymptotic Distributions of High-Dimensional Distance Correlation Inference
• Statistical quantification of confounding bias in predictive modelling
• Conditional Independence Testing in Hilbert Spaces with Applications to Functional Data Analysis
• Conditional Feature Importance for Mixed Data
• Model-based causal feature selection for general response types
• Nonparametric Conditional Local Independence Testing
• On Testing Marginal versus Conditional Independence
• Residual permutation test for regression coefficient testing
• Algorithm-agnostic significance testing in supervised learning with multimodal data
• Learning to Increase the Power of Conditional Randomization Tests
• High-dimensional Statistical Inference and Variable Selection Using Sufficient Dimension Association