Exact partial information decompositions for Gaussian systems based on dependency constraints

Abstract

The Partial Information Decomposition (PID) [arXiv:1004.2515] provides a theoretical framework to characterize and quantify the structure of multivariate information sharing. A new method (Idep) has recently been proposed for computing a two-predictor PID over discrete spaces. [arXiv:1709.06653] A lattice of maximum entropy probability models is constructed based on marginal dependency constraints, and the unique information that a particular predictor has about the target is defined as the minimum increase in joint predictor-target mutual information when that particular predictor-target marginal dependency is constrained. Here, we apply the Idep approach to Gaussian systems, for which the marginally constrained maximum entropy models are Gaussian graphical models. Closed form solutions for the Idep PID are derived for both univariate and multivariate Gaussian systems. Numerical and graphical illustrations are provided, together with practical and theoretical comparisons of the Idep PID with the minimum mutual information PID (Immi). [arXiv:1411.2832] In particular, it is proved that the Immi method generally produces larger estimates of redundancy and synergy than does the Idep method. In discussion of the practical examples, the PIDs are complemented by the use of deviance tests for the comparison of Gaussian graphical models.

arXiv comment39 pages, 9 figures, 9 tables

References in corpus (10)

• Quantifying unique information
• Exploration of synergistic and redundant information sharing in static and dynamical Gaussian systems
• A Bivariate Measure of Redundant Information
• Measuring multivariate redundant information with pointwise common change in surprisal
• Multiscale Information Decomposition: Exact Computation for Multivariate Gaussian Processes
• Expanding the Transfer Entropy to Identify Information Subgraphs in Complex Systems
• Pointwise Partial Information Decomposition using the Specificity and Ambiguity Lattices
• Secret Sharing and Shared Information
• Reconsidering unique information: Towards a multivariate information decomposition
• On extractable shared information

Cited by in corpus (5)

• Measuring Integrated Information: Comparison of Candidate Measures in Theory and Simulation
• A comparison of partial information decompositions using data from real and simulated layer 5b pyramidal cells
• Partial Information Decomposition via Deficiency for Multivariate Gaussians
• MAXENT3D_PID: An Estimator for the Maximum-entropy Trivariate Partial Information Decomposition
• Quantifying Influence and Information Transfer in a Modified Vicsek Model with Non-reciprocal Interactions