Informational and Causal Architecture of Discrete-Time Renewal Processes
arXiv:1408.6876 · doi:10.3390/e17074891
Abstract
Renewal processes are broadly used to model stochastic behavior consisting of isolated events separated by periods of quiescence, whose durations are specified by a given probability law. Here, we identify the minimal sufficient statistic for their prediction (the set of causal states), calculate the historical memory capacity required to store those states (statistical complexity), delineate what information is predictable (excess entropy), and decompose the entropy of a single measurement into that shared with the past, future, or both. The causal state equivalence relation defines a new subclass of renewal processes with a finite number of causal states despite having an unbounded interevent count distribution. We use these formulae to analyze the output of the parametrized Simple Nonunifilar Source, generated by a simple two-state hidden Markov model, but with an infinite-state epsilon-machine presentation. All in all, the results lay the groundwork for analyzing processes with infinite statistical complexity and infinite excess entropy.
arXiv comment18 pages, 9 figures, 1 table; http://csc.ucdavis.edu/~cmg/compmech/pubs/dtrp.htm
References in corpus (21)
- Computational Mechanics: Pattern and Prediction, Structure and Simplicity
- Monotonous continuous-time random walks with drift and stochastic reset events
- Time's Barbed Arrow: Irreversibility, Crypticity, and Stored Information
- Statistical Complexity of Simple 1D Spin Systems
- Anatomy of a Bit: Information in a Time Series Observation
- Structural Information in Two-Dimensional Patterns: Entropy Convergence and Excess Entropy
- Prediction, Retrodiction, and The Amount of Information Stored in the Present
- Information driven self-organization of complex robotic behaviors
- Brain, Music and non-Poisson Renewal Processes
- Bayesian Structural Inference for Hidden Processes
- Inferring hidden Markov models from noisy time sequences: a method to alleviate degeneracy in molecular dynamics
- Information Bottlenecks, Causal States, and Statistical Relevance Bases: How to Represent Relevant Information in Memoryless Transduction
- Statistics of the longest interval in renewal processes
- Information Symmetries in Irreversible Processes
- Collective Phenomena and Non-Finite State Computation in a Human Social System
- Information Anatomy of Stochastic Equilibria
- Characterization of intermittency in renewal processes: Application to earthquakes
- On Hidden Markov Processes with Infinite Excess Entropy
- Bursting transition in a linear self-exciting point process
- The Elusive Present: Hidden Past and Future Dependency and Why We Build Models
- Predictive information in a nonequilibrium critical model
Cited by in corpus (14)
- Computational Mechanics of Input-Output Processes: Structured transformations and the -transducer
- Extreme dimensionality reduction with quantum modelling
- Informational and Causal Architecture of Continuous-time Renewal and Hidden Semi-Markov Processes
- Thermodynamics of complexity and pattern manipulation
- Superior memory efficiency of quantum devices for the simulation of continuous-time stochastic processes
- Nearly Maximally Predictive Features and Their Dimensions
- Statistical Signatures of Structural Organization: The case of long memory in renewal processes
- Memory-efficient tracking of complex temporal and symbolic dynamics with quantum simulators
- Quantum adaptive agents with efficient long-term memories
- Implementing quantum dimensionality reduction for non-Markovian stochastic simulation
- Circumventing the Curse of Dimensionality in Prediction: Causal Rate-Distortion for Infinite-Order Markov Processes
- Memory in quantum dot blinking
- Quantum theory in finite dimension cannot explain every general process with finite memory
- Intrinsic computation of a Monod-Wyman-Changeux molecule