Estimation of mutual information for real-valued data with error bars and controlled bias

Abstract

Estimation of mutual information between (multidimensional) real-valued variables is used in analysis of complex systems, biological systems, and recently also quantum systems. This estimation is a hard problem, and universally good estimators provably do not exist. Kraskov et al. (PRE, 2004) introduced a successful mutual information estimation approach based on the statistics of distances between neighboring data points, which empirically works for a wide class of underlying probability distributions. Here we improve this estimator by (i) expanding its range of applicability, and by providing (ii) a self-consistent way of verifying the absence of bias, (iii) a method for estimation of its variance, and (iv) a criterion for choosing the values of the free parameter of the estimator. We demonstrate the performance of our estimator on synthetic data sets, as well as on neurophysiological and systems biology data sets.

arXiv comment10 pages, 8 figures

References in corpus (3)

• Estimating Mutual Information
• Least Dependent Component Analysis Based on Mutual Information
• Motor control by precisely timed spike patterns

Cited by in corpus (9)

• A robust estimator of mutual information for deep learning interpretability
• Data Compression and Inference in Cosmology with Self-Supervised Machine Learning
• Quantifying the informativity of emission lines to infer physical conditions in giant molecular clouds. I. Application to model predictions
• Information dynamics of price and liquidity around the 2017 Bitcoin markets crash
• Comparison of global sensitivity analysis methods for a fire spread model with a segmented characteristic
• A mutual information-based in vivo monitoring of adaptive response to targeted therapies in melanoma
• Inferring a property of a large system from a small number of samples
• The human antigen R as an actionable super-hub within the network of cancer cell persistency and plasticity
• Mutual information in changing environments: non-linear interactions, out-of-equilibrium systems, and continuously-varying diffusivities