Redundancy and synergy in dual decompositions of mutual information gain and information loss

Abstract

Williams and Beer (2010) proposed a nonnegative mutual information decomposition, based on the construction of information gain lattices, which allows separating the information that a set of variables contains about another into components interpretable as the unique information of one variable, or redundant and synergy components. In this work we extend the framework of Williams and Beer (2010) focusing on the lattices that underpin the decomposition. We generalize the type of constructible lattices and examine the relations between the terms in different lattices, for example relating bivariate and trivariate decompositions. We point out that, in information gain lattices, redundancy components are invariant across decompositions, but unique and synergy components are decomposition-dependent. Exploiting the connection between different lattices we propose a procedure to construct, in the general multivariate case, information decompositions from measures of synergy or unique information. We introduce an alternative type of mutual information decompositions based on information loss lattices, with the role and invariance properties of redundancy and synergy components exchanged with respect to gain lattices. We study the correspondence between information gain and information loss lattices and we define dual decompositions that allow overcoming the intrinsic asymmetry between invariant and decomposition-dependent components, which hinders the consistent joint characterization of synergy and redundancy.

References in corpus (9)

• Nonnegative Decomposition of Multivariate Information
• Exploration of synergistic and redundant information sharing in static and dynamical Gaussian systems
• Partial Information Decomposition as a Unified Approach to the Specification of Neural Goal Functions
• Measuring multivariate redundant information with pointwise common change in surprisal
• Synergy and redundancy in the Granger causal analysis of dynamical networks
• Multivariate Dependence Beyond Shannon Information
• Generalized Measures of Information Transfer
• Reconsidering unique information: Towards a multivariate information decomposition
• Synergy, Redundancy and Common Information

Cited by in corpus (15)

• Measuring multivariate redundant information with pointwise common change in surprisal
• Pointwise Partial Information Decomposition using the Specificity and Ambiguity Lattices
• A Novel Approach to the Partial Information Decomposition
• Invariant components of synergy, redundancy, and unique information among three variables
• Secret Sharing and Shared Information
• The Partial Entropy Decomposition: Decomposing multivariate entropy and mutual information via pointwise common surprisal
• Biological Information
• The identity of information: how deterministic dependencies constrain information synergy and redundancy
• Orders between channels and implications for partial information decomposition
• A Measure of Synergy based on Union Information
• MAXENT3D_PID: An Estimator for the Maximum-entropy Trivariate Partial Information Decomposition
• On shared and multiple information
• Quantifying multivariate redundancy with maximum entropy decompositions of mutual information
• Multivariate Partial Information Decomposition: Constructions, Inconsistencies, and Alternative Measures
• Partial Information Decomposition of Boolean Functions: a Fourier Analysis perspective