Equitability, mutual information, and the maximal information coefficient
arXiv:1301.7745 · doi:10.1073/pnas.1309933111
Abstract
Reshef et al. recently proposed a new statistical measure, the "maximal information coefficient" (MIC), for quantifying arbitrary dependencies between pairs of stochastic quantities. MIC is based on mutual information, a fundamental quantity in information theory that is widely understood to serve this need. MIC, however, is not an estimate of mutual information. Indeed, it was claimed that MIC possesses a desirable mathematical property called "equitability" that mutual information lacks. This was not proven; instead it was argued solely through the analysis of simulated data. Here we show that this claim, in fact, is incorrect. First we offer mathematical proof that no (non-trivial) dependence measure satisfies the definition of equitability proposed by Reshef et al.. We then propose a self-consistent and more general definition of equitability that follows naturally from the Data Processing Inequality. Mutual information satisfies this new definition of equitability while MIC does not. Finally, we show that the simulation evidence offered by Reshef et al. was artifactual. We conclude that estimating mutual information is not only practical for many real-world applications, but also provides a natural solution to the problem of quantifying associations in large data sets.
References in corpus (8)
- ARACNE: An Algorithm for the Reconstruction of Gene Regulatory Networks in a Mammalian Cellular Context
- Brownian distance covariance
- Entropy inference and the James-Stein estimator, with application to nonlinear gene association networks
- Adaptive Filtering Enhances Information Transmission in Visual Cortex
- A consistent multivariate test of association based on ranks of distances
- Minerva and minepy: a C engine for the MINE suite and its R, Python and MATLAB wrappers
- Equitability Analysis of the Maximal Information Coefficient, with Comparisons
- Comment on "Detecting Novel Associations In Large Data Sets" by Reshef Et Al, Science Dec 16, 2011
Cited by in corpus (55)
- Data Based Identification and Prediction of Nonlinear and Complex Dynamical Systems
- Efficient Estimation of Mutual Information for Strongly Dependent Variables
- Unsupervised and Unregistered Hyperspectral Image Super-Resolution with Mutual Dirichlet-Net
- Measuring dependence powerfully and equitably
- Information-theoretic analysis of multivariate single - cell signaling responses using SLEMI
- Consistent distribution-free -sample and independence tests for univariate random variables
- InfoBot: Transfer and Exploration via the Information Bottleneck
- BET on Independence
- Inferring gene-gene interactions and functional modules using sparse canonical correlation analysis
- Learning quantitative sequence-function relationships from massively parallel experiments
- Nonlinear dependencies on Brazilian equity network from mutual information minimum spanning trees
- Learning Disentangled Behaviour Patterns for Wearable-based Human Activity Recognition
- Exploring mutual information between IRIS spectral lines. I. Correlations between spectral lines during solar flares and within the quiet Sun
- Accurate estimation of the normalized mutual information of multidimensional data
- Estimating the Mutual Information between two Discrete, Asymmetric Variables with Limited Samples
- Calibrating cosmological simulations with implicit likelihood inference using galaxy growth observables
- Machines Learn to Infer Stellar Parameters Just by Looking at a Large Number of Spectra
- A unified data representation theory for network visualization, ordering and coarse-graining
- Insight into Ideal Shear Strength of Ni-based Dilute Alloys using First-Principles Calculations and Correlational Analysis
- Demystifying Fixed k-Nearest Neighbor Information Estimators
- Mutual Information for Explainable Deep Learning of Multiscale Systems
- Different coefficients for studying dependence
- An Empirical Study of Leading Measures of Dependence
- Theoretical Foundations of Equitability and the Maximal Information Coefficient
- MOLA: Enhancing Industrial Process Monitoring Using Multi-Block Orthogonal Long Short-Term Memory Autoencoder
- Copula Index for Detecting Dependence and Monotonicity between Stochastic Signals
- Mutual Information Gradient Estimation for Representation Learning
- Superconducting fluctuations and charge-4 plaquette state at strong coupling
- Quantum-Assisted Feature Selection for Vehicle Price Prediction Modeling
- High dimensional consistent independence testing with maxima of rank correlations
- Mutual Information Maximization for Effective Lip Reading
- On Nonsymmetric Nonparametric Measures of Dependence
- Nonlinear input feature reduction for data-based physical modeling
- A Plug-in Method for Representation Factorization in Connectionist Models
- Domain-Robust Visual Imitation Learning with Mutual Information Constraints
- A low variance consistent test of relative dependency
- BEAUTY Powered BEAST
- The Generalized Mean Information Coefficient
- Multimodal Image-to-Image Translation via Mutual Information Estimation and Maximization
- A mutual information-based in vivo monitoring of adaptive response to targeted therapies in melanoma
- Consistent distribution-free tests of association between univariate random variables
- A Copula Statistic for Measuring Nonlinear Multivariate Dependence
- A New Class of Nonsymmetric Multivariate Dependence Measures
- Network induced large correlation matrix estimation
- Adaptive Geo-Topological Independence Criterion
- fastMI: a fast and consistent copula-based estimator of mutual information
- Human Activity Recognition in an Open World
- Efficient Person Search: An Anchor-Free Approach
- Fisher exact scanning for dependency
- The dynamic importance of nodes is poorly predicted by static network features
- Neural Dependency Coding inspired Multimodal Fusion
- Fast Non-Parametric Tests of Relative Dependency and Similarity
- Equitability of Dependence Measure
- Cross-modal Image Retrieval with Deep Mutual Information Maximization
- Improving Multimodal fusion via Mutual Dependency Maximisation