A Novel Approach to the Partial Information Decomposition

Abstract

We consider the "partial information decomposition" (PID) problem, which aims to decompose the information that a set of source random variables provide about a target random variable into separate redundant, synergistic, union, and unique components. In the first part of this paper, we propose a general framework for constructing a multivariate PID. Our framework is defined in terms of a formal analogy with intersection and union from set theory, along with an ordering relation which specifies when one information source is more informative than another. Our definitions are algebraically and axiomatically motivated, and can be generalized to domains beyond Shannon information theory (such as algorithmic information theory and quantum information theory). In the second part of this paper, we use our general framework to define a PID in terms of the well-known Blackwell order, which has a fundamental operational interpretation. We demonstrate our approach on numerous examples and show that it overcomes many drawbacks associated with previous proposals.

References in corpus (10)

• Exploration of synergistic and redundant information sharing in static and dynamical Gaussian systems
• Reconciling emergences: An information-theoretic approach to identify causal emergence in multivariate data
• Pointwise Partial Information Decomposition using the Specificity and Ambiguity Lattices
• An operational information decomposition via synergistic disclosure
• Secret Sharing and Shared Information
• Unique Information and Secret Key Agreement
• Understanding Neural Networks and Individual Neuron Importance via Information-Ordered Cumulative Ablation
• Reconsidering unique information: Towards a multivariate information decomposition
• Computing the Unique Information
• Unique Informations and Deficiencies

Cited by in corpus (9)

• Decomposing causality into its synergistic, unique, and redundant components
• A comparison of partial information decompositions using data from real and simulated layer 5b pyramidal cells
• Pooling Probability Distributions and the Partial Information Decomposition
• Integrated Information Decomposition Unveils Major Structural Traits of $In$ $Silico$ and $In$ $Vitro$ Neuronal Networks
• Orders between channels and implications for partial information decomposition
• A Measure of Synergy based on Union Information
• Explicit Formula for Partial Information Decomposition
• Multivariate Partial Information Decomposition: Constructions, Inconsistencies, and Alternative Measures
• Partial information decomposition: redundancy as information bottleneck