Bits and Pieces: Understanding Information Decomposition from Part-whole Relationships and Formal Logic

Abstract

Partial information decomposition (PID) seeks to decompose the multivariate mutual information that a set of source variables contains about a target variable into basic pieces, the so called "atoms of information". Each atom describes a distinct way in which the sources may contain information about the target. In this paper we show, first, that the entire theory of partial information decomposition can be derived from considerations of elementary parthood relationships between information contributions. This way of approaching the problem has the advantage of directly characterizing the atoms of information, instead of taking an indirect approach via the concept of redundancy. Secondly, we describe several intriguing links between PID and formal logic. In particular, we show how to define a measure of PID based on the information provided by certain statements about source realizations. Furthermore, we show how the mathematical lattice structure underlying PID theory can be translated into an isomorphic structure of logical statements with a particularly simple ordering relation: logical implication. The conclusion to be drawn from these considerations is that there are three isomorphic "worlds" of partial information decomposition, i.e. three equivalent ways to mathematically describe the decomposition of the information carried by a set of sources about a target: the world of parthood relationships, the world of logical statements, and the world of antichains that was utilized by Williams and Beer in their original exposition of PID theory. We additionally show how the parthood perspective provides a systematic way to answer a type of question that has been much discussed in the PID field: whether a partial information decomposition can be uniquely determined based on concepts other than redundant information.

25 pages, 16 figures

References in corpus (5)

• Nonnegative Decomposition of Multivariate Information
• Reconciling emergences: An information-theoretic approach to identify causal emergence in multivariate data
• Introducing a differentiable measure of pointwise shared information
• An operational information decomposition via synergistic disclosure
• Secret Sharing and Shared Information

Cited by in corpus (14)

• Decomposing causality into its synergistic, unique, and redundant components
• Introducing a differentiable measure of pointwise shared information
• Decomposing past and future: Integrated information decomposition based on shared probability mass exclusions
• Quantifying high-order interdependencies on individual patterns via the local O-information: theory and applications to music analysis
• Flickering emergences: The question of locality in information-theoretic approaches to emergence
• A partial information decomposition for discrete and continuous variables
• A General Framework for Interpretable Neural Learning based on Local Information-Theoretic Goal Functions
• Pooling Probability Distributions and the Partial Information Decomposition
• A Rigorous Information-Theoretic Definition of Redundancy and Relevancy in Feature Selection Based on (Partial) Information Decomposition
• Orders between channels and implications for partial information decomposition
• A Measure of Synergy based on Union Information
• Multivariate Partial Information Decomposition: Constructions, Inconsistencies, and Alternative Measures
• Intersectional synergies: untangling irreducible effects of intersecting identities via information decomposition
• Partial Information Decomposition of Boolean Functions: a Fourier Analysis perspective