Pointwise Partial Information Decomposition using the Specificity and Ambiguity Lattices
arXiv:1801.09010 · doi:10.3390/e20040297
Abstract
What are the distinct ways in which a set of predictor variables can provide information about a target variable? When does a variable provide unique information, when do variables share redundant information, and when do variables combine synergistically to provide complementary information? The redundancy lattice from the partial information decomposition of Williams and Beer provided a promising glimpse at the answer to these questions. However, this structure was constructed using a much criticised measure of redundant information, and despite sustained research, no completely satisfactory replacement measure has been proposed. In this paper, we take a different approach, applying the axiomatic derivation of the redundancy lattice to a single realisation from a set of discrete variables. To overcome the difficulty associated with signed pointwise mutual information, we apply this decomposition separately to the unsigned entropic components of pointwise mutual information which we refer to as the specificity and ambiguity. This yields a separate redundancy lattice for each component. Then based upon an operational interpretation of redundancy, we define measures of redundant specificity and ambiguity enabling us to evaluate the partial information atoms in each lattice. These atoms can be recombined to yield the sought-after multivariate information decomposition. We apply this framework to canonical examples from the literature and discuss the results and the various properties of the decomposition. In particular, the pointwise decomposition using specificity and ambiguity satisfies a chain rule over target variables, which provides new insights into the so-called two-bit-copy example.
arXiv comment31 pages, 10 figures. (v1: preprint; v2: as accepted; v3: title corrected)
References in corpus (20)
- Quantifying unique information
- Exploration of synergistic and redundant information sharing in static and dynamical Gaussian systems
- A Bivariate Measure of Redundant Information
- Measuring multivariate redundant information with pointwise common change in surprisal
- Multiscale Information Decomposition: Exact Computation for Multivariate Gaussian Processes
- Intersection Information based on Common Randomness
- Shared Information -- New Insights and Problems in Decomposing Information in Complex Systems
- Multivariate Dependence Beyond Shannon Information
- Invariant components of synergy, redundancy, and unique information among three variables
- Redundancy and synergy in dual decompositions of mutual information gain and information loss
- Towards a Synergy-based Approach to Measuring Information Modification
- Understanding interdependency through complex information sharing
- Synergetic and redundant information flow detected by unnormalized Granger causality: application to resting state fMRI
- Redundant variables and Granger causality
- Quantifying Redundant Information in Predicting a Target Random Variable
- Secret Sharing and Shared Information
- Reconsidering unique information: Towards a multivariate information decomposition
- On extractable shared information
- Coarse-graining and the Blackwell order
- Probability Mass Exclusions and the Directed Components of Pointwise Mutual Information
Cited by in corpus (24)
- Quantifying High-order Interdependencies via Multivariate Extensions of the Mutual Information
- Bits and Pieces: Understanding Information Decomposition from Part-whole Relationships and Formal Logic
- Integrated information as a common signature of dynamical and information-processing complexity
- A Novel Approach to the Partial Information Decomposition
- An information-theoretic approach to self-organisation: Emergence of complex interdependencies in coupled dynamical systems
- Introducing a differentiable measure of pointwise shared information
- An operational information decomposition via synergistic disclosure
- A comparison of partial information decompositions using data from real and simulated layer 5b pyramidal cells
- Decomposing past and future: Integrated information decomposition based on shared probability mass exclusions
- Unique Information and Secret Key Agreement
- Generalised Measures of Multivariate Information Content
- Quantifying high-order interdependencies on individual patterns via the local O-information: theory and applications to music analysis
- Exact partial information decompositions for Gaussian systems based on dependency constraints
- Probability Mass Exclusions and the Directed Components of Pointwise Mutual Information
- Biological Information
- A General Framework for Interpretable Neural Learning based on Local Information-Theoretic Goal Functions
- Direct and Indirect Effects -- An Information Theoretic Perspective
- Partial Information Decomposition via Deficiency for Multivariate Gaussians
- Pooling Probability Distributions and the Partial Information Decomposition
- Orders between channels and implications for partial information decomposition
- MAXENT3D_PID: An Estimator for the Maximum-entropy Trivariate Partial Information Decomposition
- Explicit Formula for Partial Information Decomposition
- Multivariate Partial Information Decomposition: Constructions, Inconsistencies, and Alternative Measures
- Quantifying Influence and Information Transfer in a Modified Vicsek Model with Non-reciprocal Interactions