Prediction, Retrodiction, and The Amount of Information Stored in the Present

Abstract

We introduce an ambidextrous view of stochastic dynamical systems, comparing their forward-time and reverse-time representations and then integrating them into a single time-symmetric representation. The perspective is useful theoretically, computationally, and conceptually. Mathematically, we prove that the excess entropy--a familiar measure of organization in complex systems--is the mutual information not only between the past and future, but also between the predictive and retrodictive causal states. Practically, we exploit the connection between prediction and retrodiction to directly calculate the excess entropy. Conceptually, these lead one to discover new system invariants for stochastic dynamical systems: crypticity (information accessibility) and causal irreversibility. Ultimately, we introduce a time-symmetric representation that unifies all these quantities, compressing the two directional representations into one. The resulting compression offers a new conception of the amount of information stored in the present.

17 pages, 7 figures, 1 table; http://users.cse.ucdavis.edu/~cmg/compmech/pubs/pratisp.htm

Cited by in corpus (48)

• Weak Values are Interference Phenomena
• Anatomy of a Bit: Information in a Time Series Observation
• Demon Dynamics: Deterministic Chaos, the Szilard Map, and the Intelligence of Thermodynamic Systems
• Computational Mechanics of Input-Output Processes: Structured transformations and the $\epsilon$-transducer
• An Overview of Open-Ended Evolution: Editorial Introduction to the Open-Ended Evolution II Special Issue
• Bayesian Structural Inference for Hidden Processes
• The diminishing role of hubs in dynamical processes on complex networks
• What did Erwin Mean? The Physics of Information from the Materials Genomics of Aperiodic Crystals and Water to Molecular Information Catalysts and Life
• Chaos Forgets and Remembers: Measuring Information Creation, Destruction, and Storage
• A Closed-Form Shave from Occam's Quantum Razor: Exact Results for Quantum Compression
• Many Roads to Synchrony: Natural Time Scales and Their Algorithms
• Informational and Causal Architecture of Discrete-Time Renewal Processes
• Informational and Causal Architecture of Continuous-time Renewal and Hidden Semi-Markov Processes
• Quantum instruments as a foundation for both states and observables
• Synchronization and Control in Intrinsic and Designed Computation: An Information-Theoretic Analysis of Competing Models of Stochastic Computation
• Transient Dissipation and Structural Costs of Physical Information Transduction
• Information Symmetries in Irreversible Processes
• Nearly Maximally Predictive Features and Their Dimensions
• How Hidden are Hidden Processes? A Primer on Crypticity and Entropy Convergence
• Information Anatomy of Stochastic Equilibria
• Statistical Signatures of Structural Organization: The case of long memory in renewal processes
• Asymptotic Synchronization for Finite-State Sources
• On Hidden Markov Processes with Infinite Excess Entropy
• Spectral Simplicity of Apparent Complexity, Part I: The Nondiagonalizable Metadynamics of Prediction
• Always-On Quantum Error Tracking with Continuous Parity Measurements
• Discovering Causal Structure with Reproducing-Kernel Hilbert Space $\epsilon$-Machines
• Functional Thermodynamics of Maxwellian Ratchets: Constructing and Deconstructing Patterns, Randomizing and Derandomizing Behaviors
• The Elusive Present: Hidden Past and Future Dependency and Why We Build Models
• An Information-Theoretic Formalism for Multiscale Structure in Complex Systems
• Measurement-Induced Randomness and Structure in Controlled Qubit Processes
• The Markov Memory for Generating Rare Events
• Shortcuts to Thermodynamic Computing: The Cost of Fast and Faithful Erasure
• Diffraction Patterns of Layered Close-packed Structures from Hidden Markov Models
• Circumventing the Curse of Dimensionality in Prediction: Causal Rate-Distortion for Infinite-Order Markov Processes
• A Markov theoretic description of stacking disordered aperiodic crystals including ice and opaline silica
• The Past and the Future in the Present
• Engines for predictive work extraction from memoryful quantum stochastic processes
• Trimming the Independent Fat: Sufficient Statistics, Mutual Information, and Predictability from Effective Channel States
• Limits on Inferring the Past
• Information Accessibility and Cryptic Processes: Linear Combinations of Causal States
• The Origins of Computational Mechanics: A Brief Intellectual History and Several Clarifications
• A hidden Markov model for describing turbostratic disorder applied to carbon blacks and graphene
• Optimality and Complexity in Measured Quantum-State Stochastic Processes
• Properties of Persistent Mutual Information and Emergence
• Intrinsic computation of a Monod-Wyman-Changeux molecule
• Quantifying Emergence in terms of Persistent Mutual Information
• Ideal stochastic process modeling with post-quantum quasiprobabilistic theories
• How the Dimension of Space Affects the Products of Pre-Biotic Evolution: The Spatial Population Dynamics of Structural Complexity and The Emergence of Membranes