Invariant components of synergy, redundancy, and unique information among three variables

Abstract

In a system of three stochastic variables, the Partial Information Decomposition (PID) of Williams and Beer dissects the information that two variables (sources) carry about a third variable (target) into nonnegative information atoms that describe redundant, unique, and synergistic modes of dependencies among the variables. However, the classification of the three variables into two sources and one target limits the dependency modes that can be quantitatively resolved, and does not naturally suit all systems. Here, we extend the PID to describe trivariate modes of dependencies in full generality, without introducing additional decomposition axioms or making assumptions about the target/source nature of the variables. By comparing different PID lattices of the same system, we unveil a finer PID structure made of seven nonnegative information subatoms that are invariant to different target/source classifications and that are sufficient to construct any PID lattice. This finer structure naturally splits redundant information into two nonnegative components: the source redundancy, which arises from the pairwise correlations between the source variables, and the non-source redundancy, which does not, and relates to the synergistic information the sources carry about the target. The invariant structure is also sufficient to construct the system's entropy, hence it characterizes completely all the interdependencies in the system.

References in corpus (14)

• ARACNE: An Algorithm for the Reconstruction of Gene Regulatory Networks in a Mammalian Cellular Context
• Quantifying unique information
• Exploration of synergistic and redundant information sharing in static and dynamical Gaussian systems
• A Bivariate Measure of Redundant Information
• Partial Information Decomposition as a Unified Approach to the Specification of Neural Goal Functions
• Measuring multivariate redundant information with pointwise common change in surprisal
• Intersection Information based on Common Randomness
• Shared Information -- New Insights and Problems in Decomposing Information in Complex Systems
• Multivariate Dependence Beyond Shannon Information
• Redundancy and synergy in dual decompositions of mutual information gain and information loss
• Understanding interdependency through complex information sharing
• Synergetic and redundant information flow detected by unnormalized Granger causality: application to resting state fMRI
• On extractable shared information
• Only accessible information is useful: insights from gradient-mediated patterning

Cited by in corpus (8)

• Pointwise Partial Information Decomposition using the Specificity and Ambiguity Lattices
• Multivariate Dependence Beyond Shannon Information
• A comparison of partial information decompositions using data from real and simulated layer 5b pyramidal cells
• Quantifying how much sensory information in a neural code is relevant for behavior
• Generalised Measures of Multivariate Information Content
• The identity of information: how deterministic dependencies constrain information synergy and redundancy
• MAXENT3D_PID: An Estimator for the Maximum-entropy Trivariate Partial Information Decomposition
• Quantifying multivariate redundancy with maximum entropy decompositions of mutual information