Data Generating Process to Evaluate Causal Discovery Techniques for Time Series Data

Abstract

Going beyond correlations, the understanding and identification of causal relationships in observational time series, an important subfield of Causal Discovery, poses a major challenge. The lack of access to a well-defined ground truth for real-world data creates the need to rely on synthetic data for the evaluation of these methods. Existing benchmarks are limited in their scope, as they either are restricted to a "static" selection of data sets, or do not allow for a granular assessment of the methods' performance when commonly made assumptions are violated. We propose a flexible and simple to use framework for generating time series data, which is aimed at developing, evaluating, and benchmarking time series causal discovery methods. In particular, the framework can be used to fine tune novel methods on vast amounts of data, without "overfitting" them to a benchmark, but rather so they perform well in real-world use cases. Using our framework, we evaluate prominent time series causal discovery methods and demonstrate a notable degradation in performance when their assumptions are invalidated and their sensitivity to choice of hyperparameters. Finally, we propose future research directions and how our framework can support both researchers and practitioners.

17 pages, 9 figures, for associated code and data sets, see https://github.com/causalens/cdml-neurips2020

References in corpus (10)

• Measuring Information Transfer
• Granger causality and transfer entropy are equivalent for Gaussian variables
• CAM: Causal additive models, high-dimensional order search and penalized regression
• DAGs with NO TEARS: Continuous Optimization for Structure Learning
• Kernel Granger causality and the analysis of dynamical networks
• On the Role of Sparsity and DAG Constraints for Learning Linear DAGs
• High-recall causal discovery for autocorrelated time series with latent confounders
• Learning Sparse Nonparametric DAGs
• Necessary and sufficient conditions for causal feature selection in time series with latent common causes
• Learning DAGs without imposing acyclicity

Cited by in corpus (2)

• Boosting Synthetic Data Generation with Effective Nonlinear Causal Discovery
• Unsuitability of NOTEARS for Causal Graph Discovery