Characterising information-theoretic storage and transfer in continuous time processes

Abstract

The characterisation of information processing is an important task in complex systems science. Information dynamics is a quantitative methodology for modelling the intrinsic information processing conducted by a process represented as a time series, but to date has only been formulated in discrete time. Building on previous work which demonstrated how to formulate transfer entropy in continuous time, we give a total account of information processing in this setting, incorporating information storage. We find that a convergent rate of predictive capacity, comprised of the transfer entropy and active information storage, does not exist, arising through divergent rates of active information storage. We identify that active information storage can be decomposed into two separate quantities that characterise predictive capacity stored in a process: active memory utilisation and instantaneous predictive capacity. The latter involves prediction related to path regularity and so solely inherits the divergent properties of the active information storage, whilst the former permits definitions of pathwise and rate quantities. We formulate measures of memory utilisation for jump and neural spiking processes and illustrate measures of information processing in synthetic neural spiking models and coupled Ornstein-Uhlenbeck models. The application to synthetic neural spiking models demonstrates that active memory utilisation for point processes consists of discontinuous jump contributions (at spikes) interrupting a continuously varying contribution (relating to waiting times between spikes), complementing the behaviour previously demonstrated for transfer entropy in these processes.

arXiv comment25 pages, 2 figures

References in corpus (12)

• Measuring Information Transfer
• Local information transfer as a spatiotemporal filter for complex systems
• Transfer Entropy as a Log-likelihood Ratio
• The Organization of Intrinsic Computation: Complexity-Entropy Diagrams and the Diversity of Natural Information Processing
• Anatomy of a Bit: Information in a Time Series Observation
• Transfer entropy in continuous time, with applications to jump and neural spiking processes
• Complexity Through Nonextensivity
• A new method for choosing parameters in delay reconstruction-based forecast strategies
• Informational and Causal Architecture of Continuous-time Renewal and Hidden Semi-Markov Processes
• Structure and Randomness of Continuous-Time Discrete-Event Processes
• Information Anatomy of Stochastic Equilibria
• Statistical Signatures of Structural Organization: The case of long memory in renewal processes

Cited by in corpus (2)

• Information thermodynamics for interacting stochastic systems without bipartite structure
• Consensus between Epistemic Agents is Difficult