Discovering Causal Structure with Reproducing-Kernel Hilbert Space $\epsilon$-Machines

Abstract

We merge computational mechanics' definition of causal states (predictively-equivalent histories) with reproducing-kernel Hilbert space (RKHS) representation inference. The result is a widely-applicable method that infers causal structure directly from observations of a system's behaviors whether they are over discrete or continuous events or time. A structural representation -- a finite- or infinite-state kernel $\epsilon$-machine -- is extracted by a reduced-dimension transform that gives an efficient representation of causal states and their topology. In this way, the system dynamics are represented by a stochastic (ordinary or partial) differential equation that acts on causal states. We introduce an algorithm to estimate the associated evolution operator. Paralleling the Fokker-Plank equation, it efficiently evolves causal-state distributions and makes predictions in the original data space via an RKHS functional mapping. We demonstrate these techniques, together with their predictive abilities, on discrete-time, discrete-value infinite Markov-order processes generated by finite-state hidden Markov models with (i) finite or (ii) uncountably-infinite causal states and (iii) continuous-time, continuous-value processes generated by thermally-driven chaotic flows. The method robustly estimates causal structure in the presence of varying external and measurement noise levels and for very high dimensional data.

arXiv comment23 pages, 11 figures, 64 citations; https://team.inria.fr/comcausa/continuous-causal-states/

References in corpus (20)

• Computational Mechanics: Pattern and Prediction, Structure and Simplicity
• Quantifying Self-Organization with Optimal Predictors
• Time's Barbed Arrow: Irreversibility, Crypticity, and Stored Information
• Eigendecompositions of Transfer Operators in Reproducing Kernel Hilbert Spaces
• Nonparametric forecasting of low-dimensional dynamical systems
• Prediction, Retrodiction, and The Amount of Information Stored in the Present
• Automatic Filters for the Detection of Coherent Structure in Spatiotemporal Systems
• Operator-theoretic framework for forecasting nonlinear time series with kernel analog techniques
• Bayesian Structural Inference for Hidden Processes
• Informational and Causal Architecture of Continuous-time Renewal and Hidden Semi-Markov Processes
• Structure and Randomness of Continuous-Time Discrete-Event Processes
• Nearly Maximally Predictive Features and Their Dimensions
• Shannon Entropy Rate of Hidden Markov Processes
• Divergent Predictive States: The Statistical Complexity Dimension of Stationary, Ergodic Hidden Markov Processes
• Local Causal States and Discrete Coherent Structures
• Spectral Simplicity of Apparent Complexity, Part II: Exact Complexities and Complexity Spectra
• Fraudulent White Noise: Flat power spectra belie arbitrarily complex processes
• Towards Unsupervised Segmentation of Extreme Weather Events
• Spacetime Autoencoders Using Local Causal States
• Inference, Prediction, and Entropy-Rate Estimation of Continuous-time, Discrete-event Processes

Cited by in corpus (5)

• Nonequilibrium Statistical Mechanics and Optimal Prediction of Partially-Observed Complex Systems
• Memory in quantum dot blinking
• What Is a Pattern in Statistical Mechanics? Formalizing Structure and Patterns in One-Dimensional Spin Lattice Models with Computational Mechanics
• Exploring Predictive States via Cantor Embeddings and Wasserstein Distance
• Predictive complexity of quantum subsystems