Mixing, Ergodic, and Nonergodic Processes with Rapidly Growing Information between Blocks

Abstract

We construct mixing processes over an infinite alphabet and ergodic processes over a finite alphabet for which Shannon mutual information between adjacent blocks of length $n$ grows as $n^{\beta }$, where $\beta \in (0,1)$. The processes are a modification of nonergodic Santa Fe processes, which were introduced in the context of natural language modeling. The rates of mutual information for the latter processes are alike and also established in this paper. As an auxiliary result, it is shown that infinite direct products of mixing processes are also mixing.

21 pages

References in corpus (1)

• Variable-Length Coding of Two-Sided Asymptotically Mean Stationary Measures

Cited by in corpus (8)

• Excess entropy in natural language: present state and perspectives
• Is Natural Language a Perigraphic Process? The Theorem about Facts and Words Revisited
• On Hidden Markov Processes with Infinite Excess Entropy
• Hilberg Exponents: New Measures of Long Memory in the Process
• Consistency of the Plug-In Estimator of the Entropy Rate for Ergodic Processes
• A Preadapted Universal Switch Distribution for Testing Hilberg's Conjecture
• On a Class of Markov Order Estimators Based on PPM and Other Universal Codes
• Bounds for Algorithmic Mutual Information and a Unifilar Order Estimator