Causal inference in time series in terms of Rényi transfer entropy

Abstract

Uncovering causal interdependencies from observational data is one of the great challenges of nonlinear time series analysis. In this paper, we discuss this topic with the help of information-theoretic concept known as Rényi information measure. In particular, we tackle the directional information flow between bivariate time series in terms of Rényi transfer entropy. We show that by choosing Rényi $\alpha$ parameter appropriately we can control information that is transferred only between selected parts of underlying distributions. This, in turn, provides particularly potent tool for quantifying causal interdependencies in time series, where the knowledge of "black swan" events such as spikes or sudden jumps are of a key importance. In this connection, we first prove that for Gaussian variables, Granger causality and Rényi transfer entropy are entirely equivalent. Moreover, we also partially extend this results to heavy-tailed $\alpha$-Gaussian variables. These results allow to establish connection between autoregressive and Rényi entropy based information-theoretic approaches to data-driven causal inference. To aid our intuition we employ Leonenko et al. entropy estimator and analyze Rényi information flow between bivariate time series generated from two unidirectionally coupled Rössler systems. Notably, we find that Rényi transfer entropy not only allowed us to detect a threshold of synchronization but it also provided a non-trivial insight into the structure of a transient regime that exists between region of chaotic correlations and synchronization threshold. In addition, from Rényi transfer entropy we could reliably infer the direction of coupling - and hence causality, only for coupling strengths smaller that the onset value of transient regime, i.e. when two Rössler systems were coupled, but have not yet entered a synchronization.

References in corpus (23)

• SciPy 1.0--Fundamental Algorithms for Scientific Computing in Python
• Array Programming with NumPy
• Measuring Information Transfer
• Improved surrogate data for nonlinearity tests
• Surrogate time series
• Granger causality and transfer entropy are equivalent for Gaussian variables
• Recurrence-based time series analysis by means of complex network methods
• Analyzing Multiple Nonlinear Time Series with Extended Granger Causality
• Interdisciplinary application of nonlinear time series methods
• A class of Rényi information estimators for multidimensional densities
• Causal Network Inference by Optimal Causation Entropy
• Kolmogorov-Sinai entropy-rate vs. physical entropy
• Renyi's information transfer between financial time series
• Geometric detection of coupling directions by means of inter-system recurrence networks
• Maximum Entropy Principle in statistical inference: case for non-Shannonian entropies
• Large-scale nonlinear Granger causality: A data-driven, multivariate approach to recovering directed networks from short time-series data
• Generalized Additivity in Unitary Conformal Field Theories
• On the Role of Information Theoretic Uncertainty Relations in Quantum Theory
• Multifractal Diffusion Entropy Analysis: Optimal Bin Width of Probability Histograms
• On q-non-extensive statistics with non-Tsallisian entropy
• One-parameter class of uncertainty relations based on entropy power
• The world according to Renyi: Thermodynamics of multifractal systems
• On the uniqueness theorem for pseudo-additive entropies