Transfer Entropy as a Log-likelihood Ratio

Abstract

Transfer entropy, an information-theoretic measure of time-directed information transfer between joint processes, has steadily gained popularity in the analysis of complex stochastic dynamics in diverse fields, including the neurosciences, ecology, climatology and econometrics. We show that for a broad class of predictive models, the log-likelihood ratio test statistic for the null hypothesis of zero transfer entropy is a consistent estimator for the transfer entropy itself. For finite Markov chains, furthermore, no explicit model is required. In the general case, an asymptotic chi-squared distribution is established for the transfer entropy estimator. The result generalises the equivalence in the Gaussian case of transfer entropy and Granger causality, a statistical notion of causal influence based on prediction via vector autoregression, and establishes a fundamental connection between directed information transfer and causality in the Wiener-Granger sense.

References in corpus (1)

• Multivariate Granger Causality and Generalized Variance

Cited by in corpus (23)

• JIDT: An information-theoretic toolkit for studying the dynamics of complex systems
• Granger causality for state space models
• Structure and causality relations in a global network of financial companies
• The relation between Granger causality and directed information theory: a review
• Large-scale directed network inference with multivariate transfer entropy and hierarchical statistical testing
• Transfer entropy in continuous time, with applications to jump and neural spiking processes
• Inferring species interactions using Granger causality and convergent cross mapping
• Dynamical independence: discovering emergent macroscopic processes in complex dynamical systems
• Assessing the Significance of Directed and Multivariate Measures of Linear Dependence Between Time Series
• Assessing directionality and strength of coupling through symbolic analysis: an application to epilepsy patients
• Learning Wireless Networks' Topologies Using Asymmetric Granger Causality
• Information transfer between stock market sectors: A comparison between the USA and China
• Information flow networks of Chinese stock market sectors
• Correlations and Flow of Information between The New York Times and Stock Markets
• Identifying delayed directional couplings with symbolic transfer entropy
• Determining the number of factors in a forecast model by a random matrix test: cryptocurrencies
• Characterising information-theoretic storage and transfer in continuous time processes
• Entropy and Transfer Entropy: The Dow Jones and the build up to the 1997 Asian Crisis
• Open or Closed? Information Flow Decided by Transfer Operators and Forecastability Quality Metric
• Balanced Information Storage and Transfer in Modular Spiking Neural Networks
• Sampling distribution for single-regression Granger causality estimators
• Disentangling causal webs in the brain using functional Magnetic Resonance Imaging: A review of current approaches
• Quantum Entanglement in Time