An operational information decomposition via synergistic disclosure

Abstract

Multivariate information decompositions hold promise to yield insight into complex systems, and stand out for their ability to identify synergistic phenomena. However, the adoption of these approaches has been hindered by there being multiple possible decompositions, and no precise guidance for preferring one over the others. At the heart of this disagreement lies the absence of a clear operational interpretation of what synergistic information is. Here we fill this gap by proposing a new information decomposition based on a novel operationalisation of informational synergy, which leverages recent developments in the literature of data privacy. Our decomposition is defined for any number of information sources, and its atoms can be calculated using elementary optimisation techniques. The decomposition provides a natural coarse-graining that scales gracefully with the system's size, and is applicable in a wide range of scenarios of practical interest.

arXiv comment14 pages, 8 figures

References in corpus (13)

• Network information and connected correlations
• Quantifying unique information
• Exploration of synergistic and redundant information sharing in static and dynamical Gaussian systems
• Quantifying High-order Interdependencies via Multivariate Extensions of the Mutual Information
• A Bivariate Measure of Redundant Information
• Differential Privacy as a Mutual Information Constraint
• Measuring multivariate redundant information with pointwise common change in surprisal
• Intersection Information based on Common Randomness
• Pointwise Partial Information Decomposition using the Specificity and Ambiguity Lattices
• An information-theoretic approach to self-organisation: Emergence of complex interdependencies in coupled dynamical systems
• Understanding interdependency through complex information sharing
• Quantifying synergistic information using intermediate stochastic variables
• Unique Information and Secret Key Agreement

Cited by in corpus (10)

• Reconciling emergences: An information-theoretic approach to identify causal emergence in multivariate data
• Bits and Pieces: Understanding Information Decomposition from Part-whole Relationships and Formal Logic
• A Novel Approach to the Partial Information Decomposition
• Dynamical independence: discovering emergent macroscopic processes in complex dynamical systems
• Decomposing past and future: Integrated information decomposition based on shared probability mass exclusions
• Quantifying high-order interdependencies on individual patterns via the local O-information: theory and applications to music analysis
• An information-theoretic approach to hypergraph psychometrics
• Pooling Probability Distributions and the Partial Information Decomposition
• A Measure of Synergy based on Union Information
• Multivariate Partial Information Decomposition: Constructions, Inconsistencies, and Alternative Measures