On extractable shared information

Abstract

We consider the problem of quantifying the information shared by a pair of random variables $X_1,X_2$ about another variable $S$. We propose a new measure of shared information, called extractable shared information, that is left monotonic; that is, the information shared about $S$ is bounded from below by the information shared about $f(S)$ for any function $f$. We show that our measure leads to a new nonnegative decomposition of the mutual information $I(S;X_1{X}_2)$ into shared, complementary and unique components. We study properties of this decomposition and show that a left monotonic shared information is not compatible with a Blackwell interpretation of unique information. We also discuss whether it is possible to have a decomposition in which both shared and unique information are left monotonic.

12 pages, journal version

References in corpus (3)

• Partial Information Decomposition as a Unified Approach to the Specification of Neural Goal Functions
• Reconsidering unique information: Towards a multivariate information decomposition
• Coarse-graining and the Blackwell order

Cited by in corpus (13)

• Pointwise Partial Information Decomposition using the Specificity and Ambiguity Lattices
• Invariant components of synergy, redundancy, and unique information among three variables
• Introducing a differentiable measure of pointwise shared information
• Unique Information and Secret Key Agreement
• Understanding Neural Networks and Individual Neuron Importance via Information-Ordered Cumulative Ablation
• Generalised Measures of Multivariate Information Content
• Coarse-graining and the Blackwell order
• Exact partial information decompositions for Gaussian systems based on dependency constraints
• Biological Information
• The identity of information: how deterministic dependencies constrain information synergy and redundancy
• Orders between channels and implications for partial information decomposition
• MAXENT3D_PID: An Estimator for the Maximum-entropy Trivariate Partial Information Decomposition
• Partial information decomposition: redundancy as information bottleneck