Intersection Information based on Common Randomness

Abstract

The introduction of the partial information decomposition generated a flurry of proposals for defining an intersection information that quantifies how much of "the same information" two or more random variables specify about a target random variable. As of yet, none is wholly satisfactory. A palatable measure of intersection information would provide a principled way to quantify slippery concepts, such as synergy. Here, we introduce an intersection information measure based on the Gács-Körner common random variable that is the first to satisfy the coveted target monotonicity property. Our measure is imperfect, too, and we suggest directions for improvement.

19 pages, 5 figures

References in corpus (4)

• Nonnegative Decomposition of Multivariate Information
• A Bivariate Measure of Redundant Information
• Shared Information -- New Insights and Problems in Decomposing Information in Complex Systems
• Towards a Synergy-based Approach to Measuring Information Modification

Cited by in corpus (35)

• Exploration of synergistic and redundant information sharing in static and dynamical Gaussian systems
• Measuring multivariate redundant information with pointwise common change in surprisal
• Bits from Biology for Computational Intelligence
• Multiscale Information Decomposition: Exact Computation for Multivariate Gaussian Processes
• Pointwise Partial Information Decomposition using the Specificity and Ambiguity Lattices
• Multivariate Dependence Beyond Shannon Information
• A Novel Approach to the Partial Information Decomposition
• Invariant components of synergy, redundancy, and unique information among three variables
• Redundancy and synergy in dual decompositions of mutual information gain and information loss
• An operational information decomposition via synergistic disclosure
• Understanding interdependency through complex information sharing
• Quantifying synergistic information using intermediate stochastic variables
• Quantifying Redundant Information in Predicting a Target Random Variable
• Information flow through a model of the C. elegans klinotaxis circuit
• Decomposing past and future: Integrated information decomposition based on shared probability mass exclusions
• Generalised Measures of Multivariate Information Content
• Quantifying high-order interdependencies on individual patterns via the local O-information: theory and applications to music analysis
• Synergy as a warning sign of transitions: the case of the two-dimensional Ising model
• On extractable shared information
• Biological Information
• Synergistic information in a dynamical model implemented on the human structural connectome reveals spatially distinct associations with age
• A Framework for Causal Discovery in non-intervenable systems
• Synergy, Redundancy and Common Information
• The identity of information: how deterministic dependencies constrain information synergy and redundancy
• Orders between channels and implications for partial information decomposition
• A Measure of Synergy based on Union Information
• Quantifying multivariate redundancy with maximum entropy decompositions of mutual information
• Stripping syntax from complexity: An information-theoretical perspective on complex systems
• Explicit Formula for Partial Information Decomposition
• Some new insights into information decomposition in complex systems based on common information
• Multivariate Partial Information Decomposition: Constructions, Inconsistencies, and Alternative Measures
• Partial information decomposition: redundancy as information bottleneck
• Neural Dependency Coding inspired Multimodal Fusion
• Multiscale Partial Information Decomposition of Dynamic Processes with Short and Long-range correlations: Theory and Application to Cardiovascular Control
• An Approximation Scheme for Multivariate Information based on Partial Information Decomposition