Information Symmetries in Irreversible Processes

Abstract

We study dynamical reversibility in stationary stochastic processes from an information theoretic perspective. Extending earlier work on the reversibility of Markov chains, we focus on finitary processes with arbitrarily long conditional correlations. In particular, we examine stationary processes represented or generated by edge-emitting, finite-state hidden Markov models. Surprisingly, we find pervasive temporal asymmetries in the statistics of such stationary processes with the consequence that the computational resources necessary to generate a process in the forward and reverse temporal directions are generally not the same. In fact, an exhaustive survey indicates that most stationary processes are irreversible. We study the ensuing relations between model topology in different representations, the process's statistical properties, and its reversibility in detail. A process's temporal asymmetry is efficiently captured using two canonical unifilar representations of the generating model, the forward-time and reverse-time epsilon-machines. We analyze example irreversible processes whose epsilon-machine presentations change size under time reversal, including one which has a finite number of recurrent causal states in one direction, but an infinite number in the opposite. From the forward-time and reverse-time epsilon-machines, we are able to construct a symmetrized, but nonunifilar, generator of a process---the bidirectional machine. Using the bidirectional machine, we show how to directly calculate a process's fundamental information properties, many of which are otherwise only poorly approximated via process samples. The tools we introduce and the insights we offer provide a better understanding of the many facets of reversibility and irreversibility in stochastic processes.

32 pages, 17 figures, 2 tables; http://csc.ucdavis.edu/~cmg/compmech/pubs/pratisp2.htm

References in corpus (8)

• Information heat engine: converting information to energy by feedback control
• Anatomy of a Bit: Information in a Time Series Observation
• Many Roads to Synchrony: Natural Time Scales and Their Algorithms
• Synchronization and Control in Intrinsic and Designed Computation: An Information-Theoretic Analysis of Competing Models of Stochastic Computation
• How Hidden are Hidden Processes? A Primer on Crypticity and Entropy Convergence
• Enumerating Finitary Processes
• Equivalence of History and Generator Epsilon-Machines
• A measure of statistical complexity based on predictive information

Cited by in corpus (13)

• Above and Beyond the Landauer Bound: Thermodynamics of Modularity
• Informational and Causal Architecture of Discrete-Time Renewal Processes
• Quantum instruments as a foundation for both states and observables
• Transient Dissipation and Structural Costs of Physical Information Transduction
• General anesthesia reduces complexity and temporal asymmetry of the informational structures derived from neural recordings in Drosophila
• Causal Asymmetry in a Quantum World
• How Hidden are Hidden Processes? A Primer on Crypticity and Entropy Convergence
• Thermodynamically-Efficient Local Computation and the Inefficiency of Quantum Memory Compression
• Circumventing the Curse of Dimensionality in Prediction: Causal Rate-Distortion for Infinite-Order Markov Processes
• Trimming the Independent Fat: Sufficient Statistics, Mutual Information, and Predictability from Effective Channel States
• The Origins of Computational Mechanics: A Brief Intellectual History and Several Clarifications
• The fundamental thermodynamic bounds on finite models
• Ideal stochastic process modeling with post-quantum quasiprobabilistic theories