Pointwise Partial Information Decomposition using the Specificity and Ambiguity Lattices

Abstract

What are the distinct ways in which a set of predictor variables can provide information about a target variable? When does a variable provide unique information, when do variables share redundant information, and when do variables combine synergistically to provide complementary information? The redundancy lattice from the partial information decomposition of Williams and Beer provided a promising glimpse at the answer to these questions. However, this structure was constructed using a much criticised measure of redundant information, and despite sustained research, no completely satisfactory replacement measure has been proposed. In this paper, we take a different approach, applying the axiomatic derivation of the redundancy lattice to a single realisation from a set of discrete variables. To overcome the difficulty associated with signed pointwise mutual information, we apply this decomposition separately to the unsigned entropic components of pointwise mutual information which we refer to as the specificity and ambiguity. This yields a separate redundancy lattice for each component. Then based upon an operational interpretation of redundancy, we define measures of redundant specificity and ambiguity enabling us to evaluate the partial information atoms in each lattice. These atoms can be recombined to yield the sought-after multivariate information decomposition. We apply this framework to canonical examples from the literature and discuss the results and the various properties of the decomposition. In particular, the pointwise decomposition using specificity and ambiguity satisfies a chain rule over target variables, which provides new insights into the so-called two-bit-copy example.

31 pages, 10 figures. (v1: preprint; v2: as accepted; v3: title corrected)

References in corpus (6)

• Exploration of synergistic and redundant information sharing in static and dynamical Gaussian systems
• Multiscale Information Decomposition: Exact Computation for Multivariate Gaussian Processes
• Invariant components of synergy, redundancy, and unique information among three variables
• Secret Sharing and Shared Information
• Reconsidering unique information: Towards a multivariate information decomposition
• Probability Mass Exclusions and the Directed Components of Pointwise Mutual Information

Cited by in corpus (24)

• Quantifying High-order Interdependencies via Multivariate Extensions of the Mutual Information
• Bits and Pieces: Understanding Information Decomposition from Part-whole Relationships and Formal Logic
• Integrated information as a common signature of dynamical and information-processing complexity
• A Novel Approach to the Partial Information Decomposition
• An information-theoretic approach to self-organisation: Emergence of complex interdependencies in coupled dynamical systems
• Introducing a differentiable measure of pointwise shared information
• An operational information decomposition via synergistic disclosure
• A comparison of partial information decompositions using data from real and simulated layer 5b pyramidal cells
• Decomposing past and future: Integrated information decomposition based on shared probability mass exclusions
• Unique Information and Secret Key Agreement
• Generalised Measures of Multivariate Information Content
• Quantifying high-order interdependencies on individual patterns via the local O-information: theory and applications to music analysis
• Exact partial information decompositions for Gaussian systems based on dependency constraints
• Probability Mass Exclusions and the Directed Components of Pointwise Mutual Information
• Biological Information
• A General Framework for Interpretable Neural Learning based on Local Information-Theoretic Goal Functions
• Direct and Indirect Effects -- An Information Theoretic Perspective
• Partial Information Decomposition via Deficiency for Multivariate Gaussians
• Pooling Probability Distributions and the Partial Information Decomposition
• Orders between channels and implications for partial information decomposition
• MAXENT3D_PID: An Estimator for the Maximum-entropy Trivariate Partial Information Decomposition
• Explicit Formula for Partial Information Decomposition
• Multivariate Partial Information Decomposition: Constructions, Inconsistencies, and Alternative Measures
• Quantifying Influence and Information Transfer in a Modified Vicsek Model with Non-reciprocal Interactions