Network information and connected correlations

Abstract

Entropy and information provide natural measures of correlation among elements in a network. We construct here the information theoretic analog of connected correlation functions: irreducible $N$--point correlation is measured by a decrease in entropy for the joint distribution of $N$ variables relative to the maximum entropy allowed by all the observed $N-1$ variable distributions. We calculate the ``connected information'' terms for several examples, and show that it also enables the decomposition of the information that is carried by a population of elements about an outside source.

4 pages, 3 figures

Cited by in corpus (89)

• Weak pairwise correlations imply strongly correlated network states in a neural population
• Networks beyond pairwise interactions: structure and dynamics
• The physics of higher-order interactions in complex systems
• Are biological systems poised at criticality?
• Maximum entropy models for antibody diversity
• Searching for collective behavior in a network of real neurons
• Quantifying High-order Interdependencies via Multivariate Extensions of the Mutual Information
• Information based clustering
• Revealing networks from dynamics: an introduction
• The functional structure of cortical neuronal networks grown in vitro
• Measuring multivariate redundant information with pointwise common change in surprisal
• Energy landscape analysis of neuroimaging data
• Integrated information increases with fitness in the evolution of animats
• Stimulus-dependent maximum entropy models of neural population codes
• The use of information theory in evolutionary biology
• Multivariate Dependence Beyond Shannon Information
• Spin glass models for a network of real neurons
• The evolution of representation in simple cognitive networks
• Toward a statistical mechanics of four letter words
• Statistical mechanics of the US Supreme Court
• Robust information propagation through noisy neural circuits
• Irreducible multi-particle correlations in states without maximal rank
• Discovering Structure in High-Dimensional Data Through Correlation Explanation
• Decomposing the local arrow of time in interacting systems
• GHZ extraction yield for multipartite stabilizer states
• Faster solutions of the inverse pairwise Ising problem
• Redundancy and synergy in dual decompositions of mutual information gain and information loss
• Identification of functional information subgraphs in complex networks
• On the sufficiency of pairwise interactions in maximum entropy models of biological networks
• Understanding interdependency through complex information sharing
• An operational information decomposition via synergistic disclosure
• Generative Capacity of Probabilistic Protein Sequence Models
• Multivariate dependence and genetic networks inference
• Quantifying Redundant Information in Predicting a Target Random Variable
• Market structure explained by pairwise interactions
• Quantifying genuine multipartite correlations and their pattern complexity
• Tackling the subsampling problem to infer collective properties from limited data
• Emergence of local irreversibility in complex interacting systems
• Unique Information and Secret Key Agreement
• Input nonlinearities can shape beyond-pairwise correlations and improve information transmission by neural populations
• Separating intrinsic interactions from extrinsic correlations in a network of sensory neurons
• Introduction to correlation networks: Interdisciplinary approaches beyond thresholding
• Dynamics and Processing in Finite Self-Similar Networks
• A tractable method for describing complex couplings between neurons and population rate
• Random versus maximum entropy models of neural population activity
• Statistical pairwise interaction model of stock market
• Irreducible multiparty correlation can be created by local operations
• Ensemble Inhibition and Excitation in the Human Cortex: an Ising Model Analysis with Uncertainties
• Modeling the correlated activity of neural populations: A review
• Markovian and non-Markovian quantum measurements
• Reliability of energy landscape analysis of resting-state functional MRI data
• Pairwise Network Information and Nonlinear Correlations
• A generalization of the maximum entropy principle for curved statistical manifolds
• Role of correlations in population coding
• Polyadic Entropy, Synergy and Redundancy among Statistically Independent Processes in Nonlinear Statistical Physics with Microphysical Codependence
• An Information-Theoretic Formalism for Multiscale Structure in Complex Systems
• Information-theoretic neuro-correlates boost evolution of cognitive systems
• Information-theoretic inference of common ancestors
• Information Fragmentation, Encryption and Information Flow in Complex Biological Networks
• Synergy, Redundancy and Common Information
• Local paths to global coherence: cutting networks down to size
• When are correlations strong?
• The identity of information: how deterministic dependencies constrain information synergy and redundancy
• From statistical mechanics to information theory: understanding biophysical information-processing systems
• Resolving entropy contributions in nonequilibrium transitions
• Predicting trend reversals using market instantaneous state
• Optimized brute-force algorithms for the bifurcation analysis of a spin-glass-like neural network model
• Emergence of functional information from multivariate correlations
• Improving predictability of time series using maximum entropy methods
• Message-Passing on Hypergraphs: Detectability, Phase Transitions and Higher-Order Information
• Thermodynamics of Encoding and Encoders
• Distribution of Mutual Information in Multipartite States
• Counting solutions from finite samplings
• Cell biology: Networks, regulation, pathways
• MAXENT3D_PID: An Estimator for the Maximum-entropy Trivariate Partial Information Decomposition
• Quantum Mechanics of Consecutive Measurements
• An efficient numerical algorithm on irreducible multiparty correlations
• Correlation in states of two identical particles
• Utilizing entropy to systematically quantify the resting-condition baroreflex regulation function
• Higher-order null models as a lens for social systems
• Quantifying multivariate redundancy with maximum entropy decompositions of mutual information
• Combinatorial coding in neural populations
• Multiparticle correlation expansion of relative entropy in lattice systems
• Worldwide bilateral geopolitical interactions network inferred from national disciplinary profiles
• Multiscale Structure of More-than-Binary Variables
• Nonparametric Maximum Entropy Estimation on Information Diagrams
• Impact of triplet correlations on neural population codes
• Cooperation and dependencies in multipartite systems
• Hierarchical Quantification of Synergy in Channels