How Hidden are Hidden Processes? A Primer on Crypticity and Entropy Convergence

Abstract

We investigate a stationary process's crypticity---a measure of the difference between its hidden state information and its observed information---using the causal states of computational mechanics. Here, we motivate crypticity and cryptic order as physically meaningful quantities that monitor how hidden a hidden process is. This is done by recasting previous results on the convergence of block entropy and block-state entropy in a geometric setting, one that is more intuitive and that leads to a number of new results. For example, we connect crypticity to how an observer synchronizes to a process. We show that the block-causal-state entropy is a convex function of block length. We give a complete analysis of spin chains. We present a classification scheme that surveys stationary processes in terms of their possible cryptic and Markov orders. We illustrate related entropy convergence behaviors using a new form of foliated information diagram. Finally, along the way, we provide a variety of interpretations of crypticity and cryptic order to establish their naturalness and pervasiveness. Hopefully, these will inspire new applications in spatially extended and network dynamical systems.

18 pages, 18 figures; http://csc.ucdavis.edu/~cmg/compmech/pubs/iacp2.htm

References in corpus (3)

• Anatomy of a Bit: Information in a Time Series Observation
• Synchronization and Control in Intrinsic and Designed Computation: An Information-Theoretic Analysis of Competing Models of Stochastic Computation
• Information Symmetries in Irreversible Processes

Cited by in corpus (13)

• JIDT: An information-theoretic toolkit for studying the dynamics of complex systems
• Weak Values are Interference Phenomena
• What did Erwin Mean? The Physics of Information from the Materials Genomics of Aperiodic Crystals and Water to Molecular Information Catalysts and Life
• A Closed-Form Shave from Occam's Quantum Razor: Exact Results for Quantum Compression
• Extreme Quantum Advantage for Rare-Event Sampling
• Thermodynamics of complexity and pattern manipulation
• Information Symmetries in Irreversible Processes
• Transient Dissipation and Structural Costs of Physical Information Transduction
• Causal Asymmetry in a Quantum World
• Robust inference of memory structure for efficient quantum modelling of stochastic processes
• Spectral Simplicity of Apparent Complexity, Part I: The Nondiagonalizable Metadynamics of Prediction
• Prediction and Generation of Binary Markov Processes: Can a Finite-State Fox Catch a Markov Mouse?
• The fundamental thermodynamic bounds on finite models