Transfer entropy in continuous time, with applications to jump and neural spiking processes

Abstract

Transfer entropy has been used to quantify the directed flow of information between source and target variables in many complex systems. While transfer entropy was originally formulated in discrete time, in this paper we provide a framework for considering transfer entropy in continuous time systems, based on Radon-Nikodym derivatives between measures of complete path realizations. To describe the information dynamics of individual path realizations, we introduce the pathwise transfer entropy, the expectation of which is the transfer entropy accumulated over a finite time interval. We demonstrate that this formalism permits an instantaneous transfer entropy rate. These properties are analogous to the behavior of physical quantities defined along paths such as work and heat. We use this approach to produce an explicit form for the transfer entropy for pure jump processes, and highlight the simplified form in the specific case of point processes (frequently used in neuroscience to model neural spike trains). Finally, we present two synthetic spiking neuron model examples to exhibit the pertinent features of our formalism, namely, that the information flow for point processes consists of discontinuous jump contributions (at spikes in the target) interrupting a continuously varying contribution (relating to waiting times between target spikes). Numerical schemes based on our formalism promise significant benefits over existing strategies based on discrete time formalisms.

24 pages, 2 figures

References in corpus (6)

• Dynamical synapses causing self-organized criticality in neural networks
• JIDT: An information-theoretic toolkit for studying the dynamics of complex systems
• Local information transfer as a spatiotemporal filter for complex systems
• Second-law-like inequalities with information and their interpretations
• Synergy and redundancy in the Granger causal analysis of dynamical networks
• Entropy and Transfer Entropy: The Dow Jones and the build up to the 1997 Asian Crisis

Cited by in corpus (15)

• Semantic information, autonomous agency, and nonequilibrium statistical physics
• Large-scale directed network inference with multivariate transfer entropy and hierarchical statistical testing
• Breakdown of local information processing may underlie isoflurane anesthesia effects
• Dynamical independence: discovering emergent macroscopic processes in complex dynamical systems
• Thermodynamics and computation during collective motion near criticality
• Characterising information-theoretic storage and transfer in continuous time processes
• Modes of Information Flow in Collective Cohesion
• Influence of time delay on information exchanges between coupled linear stochastic systems
• Entropy balance and Information processing in bipartite and non-bipartite composite systems
• Information processing in a simple one-step cascade
• Feedforward and feedback influences through distinct frequency bands between two spiking-neuron networks
• Granger Causality for Compressively Sensed Sparse Signals
• Consensus between Epistemic Agents is Difficult
• Fourier-domain transfer entropy spectrum
• A Development of Continuous-Time Transfer Entropy