Relating Granger causality to directed information theory for networks of stochastic processes

Abstract

This paper addresses the problem of inferring circulation of information between multiple stochastic processes. We discuss two possible frameworks in which the problem can be studied: directed information theory and Granger causality. The main goal of the paper is to study the connection between these two frameworks. In the case of directed information theory, we stress the importance of Kramer's causal conditioning. This type of conditioning is necessary not only in the definition of the directed information but also for handling causal side information. We also show how directed information decomposes into the sum of two measures, the first one related to Schreiber's transfer entropy quantifies the dynamical aspects of causality, whereas the second one, termed instantaneous information exchange, quantifies the instantaneous aspect of causality. After having recalled the definition of Granger causality, we establish its connection with directed information theory. The connection is particularly studied in the Gaussian case, showing that Geweke's measures of Granger causality correspond to the transfer entropy and the instantaneous information exchange. This allows to propose an information theoretic formulation of Granger causality.

submitted, completely rehaul, new title, added recent references, more emphasis on general case

References in corpus (1)

• Granger causality and transfer entropy are equivalent for Gaussian variables

Cited by in corpus (10)

• Transfer Entropy as a Log-likelihood Ratio
• The relation between Granger causality and directed information theory: a review
• On directed information theory and Granger causality graphs
• Universal Estimation of Directed Information
• Transfer entropy in continuous time, with applications to jump and neural spiking processes
• Transfer Entropy Rate Through Lempel-Ziv Complexity
• Permutation Complexity and Coupling Measures in Hidden Markov Models
• Structure Learning from Time Series with False Discovery Control
• Symbolic local information transfer
• Causal conditioning and instantaneous coupling in causality graphs