Mutual Information second order expansion is the Pearson's chi-square statistic

Abstract

We show that MI connects subtly and elegantly the two best-known state-of-the-art independence test statistics: the $G^2$ and the Pearson's chi-square statistic $\ddot{I}{‡}^2$. Furthermore, we show that the MI connects directly those statistics by an elegant formula arising from a stochastic Taylor expansion of MI ($\hat{I}´$-method). MI second order term is precisely $\ddot{I}{‡}^2$ up to a scale factor. As a consequence, by this connection, the difference between $G^2$ and $\ddot{I}{‡}^2$ can be explicitly quantified.