Explicit Formula for Partial Information Decomposition
arXiv:2402.03554 · doi:10.1109/ISIT57864.2024.10619369
Abstract
Mutual information between two random variables is a well-studied notion, whose understanding is fairly complete. Mutual information between one random variable and a pair of other random variables, however, is a far more involved notion. Specifically, Shannon's mutual information does not capture fine-grained interactions between those three variables, resulting in limited insights in complex systems. To capture these fine-grained interactions, in 2010 Williams and Beer proposed to decompose this mutual information to information atoms, called unique, redundant, and synergistic, and proposed several operational axioms that these atoms must satisfy. In spite of numerous efforts, a general formula which satisfies these axioms has yet to be found. Inspired by Judea Pearl's do-calculus, we resolve this open problem by introducing the do-operation, an operation over the variable system which sets a certain marginal to a desired value, which is distinct from any existing approaches. Using this operation, we provide the first explicit formula for calculating the information atoms so that Williams and Beer's axioms are satisfied, as well as additional properties from subsequent studies in the field.
References in corpus (11)
- Quantifying unique information
- A Bivariate Measure of Redundant Information
- Reconciling emergences: An information-theoretic approach to identify causal emergence in multivariate data
- Measuring multivariate redundant information with pointwise common change in surprisal
- Intersection Information based on Common Randomness
- Shared Information -- New Insights and Problems in Decomposing Information in Complex Systems
- Pointwise Partial Information Decomposition using the Specificity and Ambiguity Lattices
- A Novel Approach to the Partial Information Decomposition
- Towards a Synergy-based Approach to Measuring Information Modification
- Quantifying Redundant Information in Predicting a Target Random Variable
- Computing the Unique Information