fastMI: a fast and consistent copula-based estimator of mutual information

Abstract

As a fundamental concept in information theory, mutual information ($MI$) has been commonly applied to quantify association between random vectors. Most existing nonparametric estimators of $MI$ have unstable statistical performance since they involve parameter tuning. We develop a consistent and powerful estimator, called fastMI, that does not incur any parameter tuning. Based on a copula formulation, fastMI estimates $MI$ by leveraging Fast Fourier transform-based estimation of the underlying density. Extensive simulation studies reveal that fastMI outperforms state-of-the-art estimators with improved estimation accuracy and reduced run time for large data sets. fastMI provides a powerful test for independence that exhibits satisfactory type I error control. Anticipating that it will be a powerful tool in estimating mutual information in a broad range of data, we develop an R package fastMI for broader dissemination.

arXiv comment5 tables, 5 figures

References in corpus (7)

• Estimating Mutual Information
• Measuring and testing dependence by correlation of distances
• Equitability, mutual information, and the maximal information coefficient
• A consistent multivariate test of association based on ranks of distances
• Topological Information Data Analysis
• Self-consistent method for density estimation
• Information estimation using nonparametric copulas