JIDT: An information-theoretic toolkit for studying the dynamics of complex systems

Abstract

Complex systems are increasingly being viewed as distributed information processing systems, particularly in the domains of computational neuroscience, bioinformatics and Artificial Life. This trend has resulted in a strong uptake in the use of (Shannon) information-theoretic measures to analyse the dynamics of complex systems in these fields. We introduce the Java Information Dynamics Toolkit (JIDT): a Google code project which provides a standalone, (GNU GPL v3 licensed) open-source code implementation for empirical estimation of information-theoretic measures from time-series data. While the toolkit provides classic information-theoretic measures (e.g. entropy, mutual information, conditional mutual information), it ultimately focusses on implementing higher-level measures for information dynamics. That is, JIDT focusses on quantifying information storage, transfer and modification, and the dynamics of these operations in space and time. For this purpose, it includes implementations of the transfer entropy and active information storage, their multivariate extensions and local or pointwise variants. JIDT provides implementations for both discrete and continuous-valued data for each measure, including various types of estimator for continuous data (e.g. Gaussian, box-kernel and Kraskov-Stoegbauer-Grassberger) which can be swapped at run-time due to Java's object-oriented polymorphism. Furthermore, while written in Java, the toolkit can be used directly in MATLAB, GNU Octave, Python and other environments. We present the principles behind the code design, and provide several examples to guide users.

arXiv comment37 pages, 4 figures

References in corpus (21)

• Measuring Information Transfer
• Estimating Mutual Information
• Granger causality and transfer entropy are equivalent for Gaussian variables
• Local information transfer as a spatiotemporal filter for complex systems
• Nonnegative Decomposition of Multivariate Information
• Model-free reconstruction of neuronal network connectivity from calcium imaging signals
• A Bivariate Measure of Redundant Information
• Differentiating information transfer and causal effect
• Least Dependent Component Analysis Based on Mutual Information
• Efficient transfer entropy analysis of non-stationary neural time series
• Transfer Entropy as a Log-likelihood Ratio
• Structure and causality relations in a global network of financial companies
• Entropy estimates of small data sets
• Automatic Filters for the Detection of Coherent Structure in Spatiotemporal Systems
• Shared Information -- New Insights and Problems in Decomposing Information in Complex Systems
• Assessing coupling dynamics from an ensemble of time series
• Complexity Through Nonextensivity
• A framework for the local information dynamics of distributed computation in complex systems
• Generalized Measures of Information Transfer
• How Hidden are Hidden Processes? A Primer on Crypticity and Entropy Convergence
• On active information storage in input-driven systems

Cited by in corpus (47)

• Reconciling emergences: An information-theoretic approach to identify causal emergence in multivariate data
• Bits from Biology for Computational Intelligence
• Large-scale directed network inference with multivariate transfer entropy and hierarchical statistical testing
• IDTxl: The Information Dynamics Toolkit xl: a Python package for the efficient analysis of multivariate information dynamics in networks
• Control of criticality and computation in spiking neuromorphic networks with plasticity
• Emotional EEG Classification using Connectivity Features and Convolutional Neural Networks
• Transfer entropy in continuous time, with applications to jump and neural spiking processes
• Decomposing causality into its synergistic, unique, and redundant components
• Large-scale nonlinear Granger causality: A data-driven, multivariate approach to recovering directed networks from short time-series data
• A framework for the local information dynamics of distributed computation in complex systems
• Inferring network properties from time series using transfer entropy and mutual information: validation of multivariate versus bivariate approaches
• Breakdown of local information processing may underlie isoflurane anesthesia effects
• Evaluation of Granger causality measures for constructing networks from multivariate time series
• Assessing the Significance of Directed and Multivariate Measures of Linear Dependence Between Time Series
• Multiscale Wavelet Transfer Entropy with Application to Corticomuscular Coupling Analysis
• A new method for choosing parameters in delay reconstruction-based forecast strategies
• netrd: A library for network reconstruction and graph distances
• Decomposing past and future: Integrated information decomposition based on shared probability mass exclusions
• Adaptive Dependency Learning Graph Neural Networks
• Causality indices for bivariate time series data: a comparative review of performance
• Determining the number of factors in a forecast model by a random matrix test: cryptocurrencies
• Information-theoretic analysis of the directional influence between cellular processes
• Modes of Information Flow in Collective Cohesion
• Precursor-driven machine learning prediction of chaotic extreme pulses in Kerr resonators
• Entropy and Transfer Entropy: The Dow Jones and the build up to the 1997 Asian Crisis
• ITENE: Intrinsic Transfer Entropy Neural Estimator
• Inter-Scale Information Flow as a Surrogate for Downward Causation That Maintains Spiral Waves
• Neuroevolution on the Edge of Chaos
• Recording and manipulation of vagus nerve electrical activity in chronically instrumented unanesthetized near term fetal sheep
• infotheory: A C++/Python package for multivariate information theoretic analysis
• infomeasure: A Comprehensive Python Package for Information Theory Measures and Estimators
• Direct and Indirect Effects -- An Information Theoretic Perspective
• Hidden Structures of Information Transport Underlying Spiral Wave Dynamics
• Multimodal pathophysiological dataset of gradual cerebral ischemia in a cohort of juvenile pigs
• A Rigorous Information-Theoretic Definition of Redundancy and Relevancy in Feature Selection Based on (Partial) Information Decomposition
• Information dynamics of price and liquidity around the 2017 Bitcoin markets crash
• Balanced Information Storage and Transfer in Modular Spiking Neural Networks
• MAXENT3D_PID: An Estimator for the Maximum-entropy Trivariate Partial Information Decomposition
• Understanding Concept Identification as Consistent Data Clustering Across Multiple Feature Spaces
• Functional Correspondences in the Human and Marmoset Visual Cortex During Movie Watching: Insights from Correlation, Redundancy, and Synergy
• Buses of Cuernavaca - an agent-based model for universal random matrix behavior minimizing mutual information
• An interview based study of pioneering experiences in teaching and learning Complex Systems in Higher Education
• Information theoretic analysis of computational models as a tool to understand the neural basis of behaviors
• Spectral Ranking of Causal Influence in Complex Systems
• Detecting causality in Plant electrical signal by a hybrid causal analysis approach
• Scale-Invariant Structures of Spiral Waves
• Plant-wide fault and disturbance screening using combined transfer entropy and eigenvector centrality analysis