Quantifying High-order Interdependencies via Multivariate Extensions of the Mutual Information

Abstract

This article introduces a model-agnostic approach to study statistical synergy, a form of emergence in which patterns at large scales are not traceable from lower scales. Our framework leverages various multivariate extensions of Shannon's mutual information, and introduces the O-information as a metric capable of characterising synergy- and redundancy-dominated systems. We develop key analytical properties of the O-information, and study how it relates to other metrics of high-order interactions from the statistical mechanics and neuroscience literature. Finally, as a proof of concept, we use the proposed framework to explore the relevance of statistical synergy in Baroque music scores.

References in corpus (16)

• Network information and connected correlations
• Nonnegative Decomposition of Multivariate Information
• Exploration of synergistic and redundant information sharing in static and dynamical Gaussian systems
• Partial Information Decomposition as a Unified Approach to the Specification of Neural Goal Functions
• Measuring multivariate redundant information with pointwise common change in surprisal
• Measuring Integrated Information: Comparison of Candidate Measures in Theory and Simulation
• Anatomy of a Bit: Information in a Time Series Observation
• Multiscale Information Decomposition: Exact Computation for Multivariate Gaussian Processes
• Expanding the Transfer Entropy to Identify Information Subgraphs in Complex Systems
• Pointwise Partial Information Decomposition using the Specificity and Ambiguity Lattices
• Multivariate Dependence Beyond Shannon Information
• An information-theoretic approach to self-organisation: Emergence of complex interdependencies in coupled dynamical systems
• Identification of functional information subgraphs in complex networks
• Understanding interdependency through complex information sharing
• Multivariate dependence and genetic networks inference
• Large Deviations Properties of Maximum Entropy Markov Chains from Spike Trains

Cited by in corpus (30)

• Networks beyond pairwise interactions: structure and dynamics
• The physics of higher-order interactions in complex systems
• Reconciling emergences: An information-theoretic approach to identify causal emergence in multivariate data
• Disentangling high-order mechanisms and high-order behaviours in complex systems
• Unveiling the higher-order organization of multivariate time series
• Integrated information as a common signature of dynamical and information-processing complexity
• Generative power of a protein language model trained on multiple sequence alignments
• An operational information decomposition via synergistic disclosure
• Towards an extended taxonomy of information dynamics via Integrated Information Decomposition
• Decomposing past and future: Integrated information decomposition based on shared probability mass exclusions
• Functional Connectome of the Human Brain with Total Correlation
• Generalised Measures of Multivariate Information Content
• Quantifying high-order interdependencies on individual patterns via the local O-information: theory and applications to music analysis
• Synergistic signatures of group mechanisms in higher-order systems
• An information-theoretic approach to hypergraph psychometrics
• A generalization of the maximum entropy principle for curved statistical manifolds
• Higher-order Organization in the Human Brain from Matrix-Based Rényi's Entropy
• Neural complexity -- Statistical-mechanical approach of human electroencephalograms
• Simultaneity of consciousness with physical reality: the key that unlocks the mind-matter problem
• Mining higher-order triadic interactions
• Pooling Probability Distributions and the Partial Information Decomposition
• Inference of time-ordered multibody interactions
• Understanding complexity via network theory: a gentle introduction
• Quantifying High-Order Interdependencies in Entangled Quantum States
• Neural Dependency Coding inspired Multimodal Fusion
• Surveying the space of descriptions of a composite system with machine learning
• MvHo-IB: Multi-View Higher-Order Information Bottleneck for Brain Disorder Diagnosis
• Partial information decomposition: redundancy as information bottleneck
• Transfer entropy and O-information to detect grokking in tensor network multi-class classification problems
• Neighbourhood topology unveils pathological hubs in the brain networks of epilepsy-surgery patients