The identity of information: how deterministic dependencies constrain information synergy and redundancy
arXiv:1711.11408 · doi:10.3390/e20030169
Abstract
Understanding how different information sources together transmit information is crucial in many domains. For example, understanding the neural code requires characterizing how different neurons contribute unique, redundant, or synergistic pieces of information about sensory or behavioral variables. Williams and Beer (2010) proposed a partial information decomposition (PID) which separates the mutual information that a set of sources contains about a set of targets into nonnegative terms interpretable as these pieces. Quantifying redundancy requires assigning an identity to different information pieces, to assess when information is common across sources. Harder et al. (2013) proposed an identity axiom stating that there cannot be redundancy between two independent sources about a copy of themselves. However, Bertschinger et al. (2012) showed that with a deterministically related sources-target copy this axiom is incompatible with ensuring PID nonnegativity. Here we study systematically the effect of deterministic target-sources dependencies. We introduce two synergy stochasticity axioms that generalize the identity axiom, and we derive general expressions separating stochastic and deterministic PID components. Our analysis identifies how negative terms can originate from deterministic dependencies and shows how different assumptions on information identity, implicit in the stochasticity and identity axioms, determine the PID structure. The implications for studying neural coding are discussed.
References in corpus (14)
- Exploration of synergistic and redundant information sharing in static and dynamical Gaussian systems
- Measuring multivariate redundant information with pointwise common change in surprisal
- Prediction of spatio-temporal patterns of neural activity from pairwise correlations
- Bits from Biology for Computational Intelligence
- Synergy and redundancy in the Granger causal analysis of dynamical networks
- Multiscale Information Decomposition: Exact Computation for Multivariate Gaussian Processes
- Multivariate Dependence Beyond Shannon Information
- Invariant components of synergy, redundancy, and unique information among three variables
- Redundancy and synergy in dual decompositions of mutual information gain and information loss
- Synergetic and redundant information flow detected by unnormalized Granger causality: application to resting state fMRI
- Secret Sharing and Shared Information
- Quantifying how much sensory information in a neural code is relevant for behavior
- Reconsidering unique information: Towards a multivariate information decomposition
- On extractable shared information