Information Flows? A Critique of Transfer Entropies

Abstract

A central task in analyzing complex dynamics is to determine the loci of information storage and the communication topology of information flows within a system. Over the last decade and a half, diagnostics for the latter have come to be dominated by the transfer entropy. Via straightforward examples, we show that it and a derivative quantity, the causation entropy, do not, in fact, quantify the flow of information. At one and the same time they can overestimate flow or underestimate influence. We isolate why this is the case and propose several avenues to alternate measures for information flow. We also address an auxiliary consequence: The proliferation of networks as a now-common theoretical model for large-scale systems, in concert with the use of transfer-like entropies, has shoehorned dyadic relationships into our structural interpretation of the organization and behavior of complex systems. This interpretation thus fails to include the effects of polyadic dependencies. The net result is that much of the sophisticated organization of complex systems may go undetected.

6 pages, 2 figures; http://csc.ucdavis.edu/~cmg/compmech/pubs/if.htm

References in corpus (7)

• Local information transfer as a spatiotemporal filter for complex systems
• Nonnegative Decomposition of Multivariate Information
• Exploration of synergistic and redundant information sharing in static and dynamical Gaussian systems
• Causation Entropy Identifies Indirect Influences, Dominance of Neighbors and Anticipatory Couplings
• Generalized Measures of Information Transfer
• Computational Mechanics of Input-Output Processes: Structured transformations and the $\epsilon$-transducer
• Identification of functional information subgraphs in complex networks

Cited by in corpus (29)

• Multivariate Dependence Beyond Shannon Information
• Towards an extended taxonomy of information dynamics via Integrated Information Decomposition
• Anatomy of Leadership in Collective Behaviour
• Marginal and Conditional Second Laws of Thermodynamics
• Estimating Conditional Transfer Entropy in Time Series using Mutual Information and Non-linear Prediction
• Local Causal States and Discrete Coherent Structures
• Modes of Information Flow in Collective Cohesion
• Fraudulent White Noise: Flat power spectra belie arbitrarily complex processes
• Simplicial cascades are orchestrated by the multidimensional geometry of neuronal complexes
• Transfer entropy dependent on distance among agents in quantifying leader-follower relationships
• Inter-Scale Information Flow as a Surrogate for Downward Causation That Maintains Spiral Waves
• A Framework for Causal Discovery in non-intervenable systems
• Identifying causal significance in three-dimensional isotropic turbulence
• An information-theoretic approach to infer the underlying interaction domain among elements from finite length trajectories in a noisy environment
• Causal network discovery by iterative conditioning: comparison of algorithms
• Hidden Structures of Information Transport Underlying Spiral Wave Dynamics
• Information Fragmentation, Encryption and Information Flow in Complex Biological Networks
• Can Transfer Entropy Infer Information Flow in Neuronal Circuits for Cognitive Processing?
• Ambiguity Rate of Hidden Markov Processes
• Consensus between Epistemic Agents is Difficult
• Causal influence in linear response models
• Buses of Cuernavaca - an agent-based model for universal random matrix behavior minimizing mutual information
• Modes of Information Flow
• The dynamic importance of nodes is poorly predicted by static network features
• Anatomy of a Spin: The Information-Theoretic Structure of Classical Spin Systems
• Quantifying Influence and Information Transfer in a Modified Vicsek Model with Non-reciprocal Interactions
• Fluctuation-response theorem for Kullback-Leibler divergences to quantify causation
• An axiomatic measure of one-way quantum information
• Correlated structural evolution within multiplex networks