Reconsidering unique information: Towards a multivariate information decomposition

Abstract

The information that two random variables $Y$, $Z$ contain about a third random variable $X$ can have aspects of shared information (contained in both $Y$ and $Z$), of complementary information (only available from $(Y,Z)$ together) and of unique information (contained exclusively in either $Y$ or $Z$). Here, we study measures $\widetilde{SI}$ of shared, $\widetilde{UI}$ unique and $\widetilde{CI}$ complementary information introduced by Bertschinger et al., which are motivated from a decision theoretic perspective. We find that in most cases the intuitive rule that more variables contain more information applies, with the exception that $\widetilde{SI}$ and $\widetilde{CI}$ information are not monotone in the target variable $X$. Additionally, we show that it is not possible to extend the bivariate information decomposition into $\widetilde{SI}$, $\widetilde{UI}$ and $\widetilde{CI}$ to a non-negative decomposition on the partial information lattice of Williams and Beer. Nevertheless, the quantities $\widetilde{UI}$, $\widetilde{SI}$ and $\widetilde{CI}$ have a well-defined interpretation, even in the multivariate setting.

5 pages, 1 figure, submitted to ISIT 2014

References in corpus (3)

• Quantifying unique information
• A Bivariate Measure of Redundant Information
• Shared Information -- New Insights and Problems in Decomposing Information in Complex Systems

Cited by in corpus (14)

• Quantifying unique information
• Measuring multivariate redundant information with pointwise common change in surprisal
• Pointwise Partial Information Decomposition using the Specificity and Ambiguity Lattices
• A Novel Approach to the Partial Information Decomposition
• Redundancy and synergy in dual decompositions of mutual information gain and information loss
• Secret Sharing and Shared Information
• Generalised Measures of Multivariate Information Content
• On extractable shared information
• Computing the Unique Information
• Exact partial information decompositions for Gaussian systems based on dependency constraints
• Unique Informations and Deficiencies
• The identity of information: how deterministic dependencies constrain information synergy and redundancy
• Orders between channels and implications for partial information decomposition
• Unique Information and Secret Key Decompositions